Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Hub install problems #1

Open
paul94010 opened this issue Oct 24, 2014 · 20 comments
Open

API Hub install problems #1

paul94010 opened this issue Oct 24, 2014 · 20 comments

Comments

@paul94010
Copy link

After hearing about the module at DrupalCon Amsterdam, I down loaded it and tried to install it. When activating the module I got this fatal error:

Fatal error: Class 'apihub_handler' not found in /Users/paul/Sites/BackstageSept2014/sites/all/modules/apihub-7.x-1.x/apihub/includes/oauthconnector.inc on line 86

I am using the latest distro of Commerce Kickstart. Let me know if you need more information.

@Decipher
Copy link
Member

Hi Paul,

So I have had this error myself, and I really should fix it. Here's a simple workaround:

  1. Open apihub/includes/oauthconnector.inc
  2. Comment out line #86-113 (the whole class)
  3. Install the module
  4. Uncomment out line #86-113

I have a few fixes from a few days ago that I haven't yet pushed, will try to get these up in the next 24 hours as long as a fix for this issue.

@paul94010
Copy link
Author

I am finally able to get back working with your module. I now have it installed ok but I can’t seem to configure it. Do you have any documentation at all on how to use the module. Maybe the slides you showed at Amsterdam. (I can’t seem to find the materials on the Drupal website.)

Thanks.

Paul

On 23 Oct 2014, at 5:58 PM, Stuart Clark [email protected] wrote:

Hi Paul,

So I have had this error myself, and I really should fix it. Here's a simple workaround:

Open apihub/includes/oauthconnector.inc
Comment out line #86-113 (the whole class)
Install the module
Uncomment out line #86-113
I have a few fixes from a few days ago that I haven't yet pushed, will try to get these up in the next 24 hours as long as a fix for this issue.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

Decipher commented Dec 4, 2014

Hi Paul,

There is no documentation yet, nor slides or videos. I do intend to make a video demonstrating the module and how to use the module as soon as I've got something more to demonstrate. I have been working on the module of late so I do hope I can move forward with something soon.

In the meantime, the usage is essentially:

  • Define a new API
  • Define a new resource in the API
    • Add the path for the resource (/something)
    • Add the method of the resource (POST, GET, etc)
    • Save (because of bugs)
    • Edit resource
    • Add required/optional parameters
    • Save
    • Test the resource (best with Devel module)
    • Edit resource
    • Add results (based on what you saw in the test or the documentation)
    • Save
  • Invoke via Rules or Views

The work I'm currently doing is to improve the Rules implementation, as currently you need to create a handler, but that's going to be make optional. Once that's done, I want to finish the Views integration.

I'm happy to jump on IRC or similar at some stage and walk you through anything you need help with.

@paul94010
Copy link
Author

Stuart, thanks a lot. I was able to make progress with this. I still can’t get the data right but I am connecting with the external web service. My problem now is sorting of the data parameters.

I do want to confirm that JSON is a valid format for the module?

Paul
On 4 Dec 2014, at 3:29 PM, Stuart Clark [email protected] wrote:

Hi Paul,

There is no documentation yet, nor slides or videos. I do intend to make a video demonstrating the module and how to use the module as soon as I've got something more to demonstrate. I have been working on the module of late so I do hope I can move forward with something soon.

In the meantime, the usage is essentially:

Define a new API
Add the endpoint for said URL (http://exmaple.com/api http://exmaple.com/api)
Define a new resource in the API
Add the path for the resource (/something)
Add the method of the resource (POST, GET, etc)
Save (because of bugs)
Edit resource
Add required/optional parameters
Save
Test the resource (best with Devel module)
Edit resource
Add results (based on what you saw in the test or the documentation)
Save
Invoke via Rules or Views
The work I'm currently doing is to improve the Rules implementation, as currently you need to create a handler, but that's going to be make optional. Once that's done, I want to finish the Views integration.

I'm happy to jump on IRC or similar at some stage and walk you through anything you need help with.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

Hi Paul,

I have definitely been able to communicate with JSON based REST APIs. What's the API you're trying to connect with? Assuming it's public facing, I might be able to help.

@paul94010
Copy link
Author

Stuart, I am trying to extract data from this wikimedia source:

http://imslp.org/wiki/IMSLP:API

I have started with the open (public) API.

http://imslp.org/imslpscripts/API.ISCR.php?account=worklist/disclaimer=accepted/sort=id/type=1/start=0/retformat=json

I am still wait for permissions to access the full API but while waiting for approval, I am trying to get the basics set up in my applications.

The use case is:

User enters a composer’s last name.
the API returns the list of pieces the composer wrote along with details like number of each instrument required by the piece.

So far I am am getting 404 errors on all my tries.

Paul

On 9 Dec 2014, at 5:59 PM, Stuart Clark [email protected] wrote:

Hi Paul,

I have definitely been able to communicate with JSON based REST APIs. What's the API you're trying to connect with? Assuming it's public facing, I might be able to help.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

So the problem is that the API is returning the content-type header of 'text/plain', so API Hub isn't trying to decode the JSON.

Ideally the API should be returning the correct headers, but as it's not it's probably not the worst thing in the world to add the ability to force a a specific behaviour regardless of the headers.

I'll look into it ASAP.

In the mean time you can copy the switch case for 'json' (line 90-92 of apihub/apihub/includes/apihub.inc) and paste it in the same switch statement and simply change 'json' to 'plain' in the case statement.

@paul94010
Copy link
Author

Stuart, a friend has downloaded the data on wiki to a true JSON format on Cloudiant:

https://imslp.cloudant.com/dashboard.html#/database/composition/_design/Composition/_view/all https://imslp.cloudant.com/dashboard.html#/database/composition/_design/Composition/_view/all

User name: imslp
password: imslpFunScoreTempest

When I try to access the Cloudiant API for this data:

https://imslp.cloudant.com/composition/_design/Composition/_view/all https://imslp.cloudant.com/composition/_design/Composition/_view/all

by adding a resource in API Hub, I get the following ajax error:

An AJAX HTTP error occurred.
HTTP Result Code: 404
Debugging information follows.
Path: /admin/structure/apihub/list//resources/list//field/parameters
StatusText: Not Found
ResponseText:
404 error | SF Civic Symphony Backstage

I think this has nothing to do with your module but I am not sure what is causing it. Just thought you would like to know.

Here is an example of the JSON on the Cloudiant service:

"id": "03e45d301bc303dbc21cd8dfd800617f",
"key": null,
"value": {
"_id": "03e45d301bc303dbc21cd8dfd800617f",
"_rev": "7-c8b890224f7468e0d9f4a088908e8160",
"composerName": "Collections",
"composerCategoryId": "/wiki/Category:Collections",
"compositionName": "21 Chansons musicales esleves de plusieurs livres, Livre 3 (Attaingnant, Pierre)",
"compositionCategoryId": "/wiki/21_Chansons_musicales_esleves_de_plusieurs_livres,Livre_3(Attaingnant,_Pierre)"

I hope that helps.

Paul

On 9 Dec 2014, at 6:29 PM, Stuart Clark [email protected] wrote:

So the problem is that the API is returning the content-type header of 'text/plain', so API Hub isn't trying to decode the JSON.

Ideally the API should be returning the correct headers, but as it's not it's probably not the worst thing in the world to add the ability to force a a specific behaviour regardless of the headers.

I'll look into it ASAP.

In the mean time you can copy the switch case for 'json' (line 90-92 of apihub/apihub/includes/apihub.inc) and paste it in the same switch statement and simply change 'json' to 'plain' in the case statement.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

That is a known issue, and one of my top must do's. Essentially the issue is that when you create a resource you need to save it before adding parameters, as it doesn't have the machine name of the resource to use in the url (hence the //). At least, that's what it looks like.

@paul94010
Copy link
Author

At the risk of annoying you further, here are the instructions for the Cloudiant API:

https://docs.cloudant.com/tutorials/basics/httprest/index.html https://docs.cloudant.com/tutorials/basics/httprest/index.html

This interface is more suited to what I want, if I can get it to work.

Paul

On 9 Dec 2014, at 6:29 PM, Stuart Clark [email protected] wrote:

So the problem is that the API is returning the content-type header of 'text/plain', so API Hub isn't trying to decode the JSON.

Ideally the API should be returning the correct headers, but as it's not it's probably not the worst thing in the world to add the ability to force a a specific behaviour regardless of the headers.

I'll look into it ASAP.

In the mean time you can copy the switch case for 'json' (line 90-92 of apihub/apihub/includes/apihub.inc) and paste it in the same switch statement and simply change 'json' to 'plain' in the case statement.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

I will endeavour to look at it more this evening. Lunch break is over for now :)

But don't worry about annoying me, I need people to test the module and report issues so that I can get it ready for release and make it super simple to use.

@paul94010
Copy link
Author

Got it. That fixed the ajax error.

On 9 Dec 2014, at 7:09 PM, Stuart Clark [email protected] wrote:

That is a known issue, and one of my top must do's. Essentially the issue is that when you create a resource you need to save it before adding parameters, as it doesn't have the machine name of the resource to use in the url (hence the //). At least, that's what it looks like.


Reply to this email directly or view it on GitHub #1 (comment).

@paul94010
Copy link
Author

I am making progress with the Cloudiant site with real JSON formatting. I have a question about the forms for adding resources:

When adding resources, what is intended use of the “Available Options” field?

Here is screen shot:

Also, if I just try to “Test” this data, I get this AJAX error:

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /system/ajax
StatusText: OK
ResponseText:
Warning: Unknown: Input variables exceeded 2000. To increase the limit change max_input_vars in php.ini. in Unknown on line 0
[{"command":"settings","settings":{"basePath":"/","pathPrefix":"","ajaxPageState":{"theme":"omega_kickstart","theme_token":"XvvApkUqlc966XWqyk4LmhDMl-LIgHdP4aLymSSZXu8"},"overlay":{"paths":{"admin":"commerceguys-marketplace\ncommerceguys-marketplace/addons\nuser//addressbook//create\nuser//addressbook//edit/\nuser//addressbook//delete/\ncommerce_backoffice/order/\nimport\nimport/\nnode//import\nnode//delete-items\nnode//log\nnode//edit\nnode//delete\nnode//revisions\nnode//revisions//revert\nnode//revisions//delete\nnode/add\nnode/add/\noverlay/dismiss-message\nadmin\nadmin/\nbatch\ntaxonomy/term//edit\nuser//cancel\nuser//edit\nuser//edit/\nnode//display\nuser//display\ntaxonomy/term//display\ncommerce_backoffice/variations/\ndevel/\nnode//devel\nnode//devel/\ncomment//devel\ncomment//devel/\nuser//devel\nuser//devel/\ntaxonomy/term//devel\ntaxonomy/term//devel/","non_admin":"admin/structure/block/demo/*\nadmin/reports/status/php"},"pathPrefixes":[],"ajaxCallback":"overlay-ajax"}},"merge":true},{"command":"insert","method":null,"selector":null,"data":"\u003Cdiv id=\u0022test-output-wrapper\u0022\u003E\u003Cdiv id=\u0022edit-output--2\u0022 class=\u0022form-item form-type-item\u0022\u003E\n \u003Clabel for=\u0022edit-output--2\u0022\u003EOutput \u003C/label\u003E\n \u003Cpre\u003E\u003C/pre\u003E\n\u003C/div\u003E\n\u003C/div\u003E","settings":null},{"command":"insert","method":"prepend","selector":null,"data":"","settings":null}]
On 9 Dec 2014, at 7:06 PM, BassCleff94010 [email protected] wrote:

Stuart, a friend has downloaded the data on wiki to a true JSON format on Cloudiant:

https://imslp.cloudant.com/dashboard.html#/database/composition/_design/Composition/_view/all https://imslp.cloudant.com/dashboard.html#/database/composition/_design/Composition/_view/all

User name: imslp
password: imslpFunScoreTempest

When I try to access the Cloudiant API for this data:

https://imslp.cloudant.com/composition/_design/Composition/_view/all https://imslp.cloudant.com/composition/_design/Composition/_view/all

by adding a resource in API Hub, I get the following ajax error:

An AJAX HTTP error occurred.
HTTP Result Code: 404
Debugging information follows.
Path: /admin/structure/apihub/list//resources/list//field/parameters
StatusText: Not Found
ResponseText:
404 error | SF Civic Symphony Backstage

I think this has nothing to do with your module but I am not sure what is causing it. Just thought you would like to know.

Here is an example of the JSON on the Cloudiant service:

"id": "03e45d301bc303dbc21cd8dfd800617f",
"key": null,
"value": {
"_id": "03e45d301bc303dbc21cd8dfd800617f",
"_rev": "7-c8b890224f7468e0d9f4a088908e8160",
"composerName": "Collections",
"composerCategoryId": "/wiki/Category:Collections",
"compositionName": "21 Chansons musicales esleves de plusieurs livres, Livre 3 (Attaingnant, Pierre)",
"compositionCategoryId": "/wiki/21_Chansons_musicales_esleves_de_plusieurs_livres,Livre_3(Attaingnant,_Pierre)"

I hope that helps.

Paul

On 9 Dec 2014, at 6:29 PM, Stuart Clark <[email protected] mailto:[email protected]> wrote:

So the problem is that the API is returning the content-type header of 'text/plain', so API Hub isn't trying to decode the JSON.

Ideally the API should be returning the correct headers, but as it's not it's probably not the worst thing in the world to add the ability to force a a specific behaviour regardless of the headers.

I'll look into it ASAP.

In the mean time you can copy the switch case for 'json' (line 90-92 of apihub/apihub/includes/apihub.inc) and paste it in the same switch statement and simply change 'json' to 'plain' in the case statement.


Reply to this email directly or view it on GitHub #1 (comment).

@paul94010
Copy link
Author

Stuart, here is the set up screen I am using in API Hub for the Cloudiant database:

On 9 Dec 2014, at 7:14 PM, BassCleff94010 [email protected] wrote:

Got it. That fixed the ajax error.

On 9 Dec 2014, at 7:09 PM, Stuart Clark <[email protected] mailto:[email protected]> wrote:

That is a known issue, and one of my top must do's. Essentially the issue is that when you create a resource you need to save it before adding parameters, as it doesn't have the machine name of the resource to use in the url (hence the //). At least, that's what it looks like.


Reply to this email directly or view it on GitHub #1 (comment).

@Decipher
Copy link
Member

Hi Paul,

Github doesn't get screenshots from emails, so neither came through. But in response to one of your questions:

"Available Options" is for when the variable takes a limited set of options, so the Rules UI would provide that parameter as a Select box instead of a Textfield.

@paul94010
Copy link
Author

Stuart, before I forget the details, I am documenting my progress so far in using API Hub for my project. I hope it helps as you develop the module.

As a reminder, I am trying to add data from a wikimedia site (www.imslp.org http://www.imslp.org/) to my Drupal Commerce site. I am able to get the integration to basically work with the site except that there are almost 300,000 rows in the IMSLP database and the code blows up with a memory error.

The problem is that feeds is trying to import all the data, which is not what I need.

What I need is for the user to enter “Composer last name” and their guess at the “title” of the piece, have IMSLP’s internal google search engine retrieve the best answer. (In my experience this is always the first return item.) Then have the user select their choice for the google search returned data. At that point, I want to map the IMSLP data to the node fields.

I think the last step is possible now using your module. (I am also able to do the mapping in Feeds.) I need to figure out how to make it work in the above scenario.

My best idea so far is to just add a link in a page on my site that includes the IMSLP sites search field. (I haven’t been able to find a URL that will do that yet.)

Then show the IMSLP search results page on my site, in a new blank page.

When the user clicks on their selected search result, then I map the data to my sites node fields. (I am not sure how to do this yet. Maybe there is way to just access use the API Hub parser?)

Context for the Use Case: A musician suggests a musical piece that he/she wants to play with other musicians. Almost all pieces are listed on the IMSLP database, some with scores. The data is sometimes very complicated involving up to 70 people with a symphony orchestra. So its only practical to get all that detail from the IMSLP site.

In looking on the web, I believe my Use Case is not all that unique. There are many posts asking for a way to selectively add products from a third party web site. (Think a retailer who is selling products from a wholesaler. Its not helpful or sensible to carry the wholesalers inventory in the retailers database.)

So my question is, am I trying to use your module incorrectly?

If not, do you know of anyone who has implemented a simple query/response web services interface for Drupal?

Paul

@paul94010
Copy link
Author

Stuart, I ran into a bug:

After creating an API with URL and a resource with the path/authentication parameters, I am unable to edit the path. I have to create a new resource.

Paul

On 13 Dec 2014, at 11:18 AM, Paul Grantham [email protected] wrote:

Stuart, before I forget the details, I am documenting my progress so far in using API Hub for my project. I hope it helps as you develop the module.

As a reminder, I am trying to add data from a wikimedia site (www.imslp.org http://www.imslp.org/) to my Drupal Commerce site. I am able to get the integration to basically work with the site except that there are almost 300,000 rows in the IMSLP database and the code blows up with a memory error.

The problem is that feeds is trying to import all the data, which is not what I need.

What I need is for the user to enter “Composer last name” and their guess at the “title” of the piece, have IMSLP’s internal google search engine retrieve the best answer. (In my experience this is always the first return item.) Then have the user select their choice for the google search returned data. At that point, I want to map the IMSLP data to the node fields.

I think the last step is possible now using your module. (I am also able to do the mapping in Feeds.) I need to figure out how to make it work in the above scenario.

My best idea so far is to just add a link in a page on my site that includes the IMSLP sites search field. (I haven’t been able to find a URL that will do that yet.)

Then show the IMSLP search results page on my site, in a new blank page.

When the user clicks on their selected search result, then I map the data to my sites node fields. (I am not sure how to do this yet. Maybe there is way to just access use the API Hub parser?)

Context for the Use Case: A musician suggests a musical piece that he/she wants to play with other musicians. Almost all pieces are listed on the IMSLP database, some with scores. The data is sometimes very complicated involving up to 70 people with a symphony orchestra. So its only practical to get all that detail from the IMSLP site.

In looking on the web, I believe my Use Case is not all that unique. There are many posts asking for a way to selectively add products from a third party web site. (Think a retailer who is selling products from a wholesaler. Its not helpful or sensible to carry the wholesalers inventory in the retailers database.)

So my question is, am I trying to use your module incorrectly?

If not, do you know of anyone who has implemented a simple query/response web services interface for Drupal?

Paul

@Decipher
Copy link
Member

Hi Paul,

I'll come back to the big comment ASAP, as it requires some reading time before I can respond to it, but for the bug, that's a simple answer: That's by design.

The reason you can't change the path is because the path and the method (POST, GET, etc) are used to generate the unique id of a resource, as they are static, a path and it's method should never change.

However, seeing the imslp API, I can understand why this could be slightly tricky, as it puts it's parameters straight into the URL, which does of course make things a little bit trickier. While some other APIs do this as well, it's generally only one import parameter, and the rest are process as data passed to the URL.

I will have to have a bit more of a think on this, as I still think the approach that has been taken is correct, but rather maybe there should be a better way to deal with odd APIs.

@Decipher
Copy link
Member

Hi Paul,

I wanted to give you a heads up that the last two commits are part of a fairly major restructure of the underlying architecture of the module. Primarily, it is moving to a more OOP approach, which will simplify things for future development.

What it means for you (and other early adopters) is that some functionality may currently be broken, and that any feature exports of API Hub resources will be broken. If you do have any exported API Hub resources, the fix is incredibly simple, simple change all instances of $resource = new stdClass(); to $resource = new apihubResource();.

I hope this doesn't cause any major inconveniences, but be assured that I am doing my best to get this module stabilised and ready for a release ASAP! I am actively using it in a big project in some pretty amazing ways, which is what prompted the change.

@paul94010
Copy link
Author

Thanks for letting me know. I have put my project on hold because of more important priorities. I look forward to using the new version. I’ll let you know how it works for me.

Paul
On 28 Jan 2015, at 12:22 PM, Stuart Clark [email protected] wrote:

Hi Paul,

I wanted to give you a heads up that the last two commits are part of a fairly major restructure of the underlying architecture of the module. Primarily, it is moving to a more OOP approach, which will simplify things for future development.

What it means for you (and other early adopters) is that some functionality may currently be broken, and that any feature exports of API Hub resources will be broken. If you do have any exported API Hub resources, the fix is incredibly simple, simple change all instances of $resource = new stdClass(); to $resource = new apihubResource();.

I hope this doesn't cause any major inconveniences, but be assured that I am doing my best to get this module stabilised and ready for a release ASAP! I am actively using it in a big project in some pretty amazing ways, which is what prompted the change.


Reply to this email directly or view it on GitHub #1 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants