API

CAUTION: Using API during a running crawl may be unresponsive at times and slow down the crawl because it may occasionally activate creation of optimized tables for certain queries, which take more time for bigger crawls. It is best to wait for the crawl to finish, and then use the API.

API URL is of the folowing form:

http://{IP}:{PORT}/u/?id={JOB_ID}&a={ACTION_ID}&h={JOB_HASH_CODE}[&q1={QUERY_STRING_1}][&q2={QUERY_STRING_2}][&l={LIMIT_RESULTS}][&o={OFFSET}]

IP, PORT, JOB_ID and JOB_HASH_CODE are provided at the job's dashboard page upon job creation. Issuing DELETE RESULTS then START may assign a new IP. Depending on the ACTION_ID issued, QUERY_STRING_1 and QUERY_STRING_2 may be mandatory or not. OFFSET and LIMIT_RESULTS are always optional.

parameterdescriptionmandatory or optionaltype
aACTION_IDmandatoryinteger
idJOB_IDmandatoryinteger
q1QUERY_STRING_1mandatory for some actionsURL encoded string (URL or query phrase)
q2QUERY_STRING_2mandatory for some actionsURL encoded string (URL or query phrase)
lLIMIT_RESULTSoptionalinteger (1000 default & maximum)
oOFFSEToptional for some actionsinteger
hJOB_HASH_CODEmandatorystring

[] placing QUERY_STRING_1 within square brackets [] in some commands results in exact query match

If OFFSET is specified, then instead of latest results, results starting with specified OFFSET id will be retrieved. This way, user can navigate the whole set of results through the API.

Response is JSON formatted output. Every command having 'Page' or 'Title' in name, if it does not return 'on_topic' field, all results are by default on_topic! Pages table contains only on-topic pages, so every query over that table by default includes only on-topic results. Queries that include edges as well contain on_topic field for the target/source node.

ACTION_IDdescriptionmandatory parametersoptional parameters
83most relevant domains
80domains with most linking domains
79domains with most incoming links
82domains with most domains linked
81domains with most outgoing links
121tld statistics
24keyword ideas
118keywords from example.com&q1=http%3A%2F%2Fwww.example.com%2F
119atext keywords to example.com&q1=http%3A%2F%2Fwww.example.com%2F
120atext keywords from example.com&q1=http%3A%2F%2Fwww.example.com%2F
21atext
76atext to example.com&q1=http%3A%2F%2Fwww.example.com%2F
23atext to http://www.example.com/&q1=http%3A%2F%2Fwww.example.com%2F
43query phrase in atext&q1=query+phrase
98query phrase in atext to example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F
47query phrase in atext to http://www.example.com/&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F
97query phrase in atext from example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F
48query phrase in title&q1=query+phrase
99query phrase in title from example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F
37most linked pages
90most linked pages from example.com&q1=http%3A%2F%2Fwww.example.com%2F
28hubs
84hubs from example.com&q1=http%3A%2F%2Fwww.example.com%2F
50popular pages
114popular prospect pages
104popular pages pointing to example.com&q1=http%3A%2F%2Fwww.example.com%2F
59popular pages pointing to http://www.example.com/&q1=http%3A%2F%2Fwww.example.com%2F
103popular pages from example.com&q1=http%3A%2F%2Fwww.example.com%2F
117popular prospect pages from example.com&q1=http%3A%2F%2Fwww.example.com%2F
51popular pages linking with query phrase in atext&q1=query+phrase[]
101popular pages linking to example.com with query phrase in atext&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
54popular pages linking to http://www.example.com/ with query phrase in atext&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
100popular pages from example.com linking with query phrase in atext&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
55popular pages with query phrase in title&q1=query+phrase[]
115popular prospect pages with query phrase in title&q1=query+phrase[]
102popular pages with query phrase in title from example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
116popular prospect pages with query phrase in title from example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
60urls with query phrase in atext&q1=query+phrase[]
106urls with query phrase in atext to example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
64urls with query phrase in atext to http://www.example.com/&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
105urls with query phrase in atext from example.com&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
62urls with query phrase in atext from http://www.example.com/&q1=query+phrase&q2=http%3A%2F%2Fwww.example.com%2F[]
65urls with query phrase in title&q1=query+phrase
78domains with most semantic flow to example.com&q1=http%3A%2F%2Fwww.example.com%2F
77domains with most semantic flow from example.com&q1=http%3A%2F%2Fwww.example.com%2F
122tld semantic flows to example.com&q1=http%3A%2F%2Fwww.example.com%2F
123tld semantic flows from example.com&q1=http%3A%2F%2Fwww.example.com%2F
91number inbound links to example.com&q1=http%3A%2F%2Fwww.example.com%2F
92number inbound domains to example.com&q1=http%3A%2F%2Fwww.example.com%2F
93number outbound links from example.com&q1=http%3A%2F%2Fwww.example.com%2F
94number outbound domains from example.com&q1=http%3A%2F%2Fwww.example.com%2F
108example.com rank&q1=http%3A%2F%2Fwww.example.com%2F
107similar domains to example.com&q1=http%3A%2F%2Fwww.example.com%2F
88links to example.com&q1=http%3A%2F%2Fwww.example.com%2F
36links to http://www.example.com/&q1=http%3A%2F%2Fwww.example.com%2F
95links from example.com&q1=http%3A%2F%2Fwww.example.com%2F
41links from http://www.example.com/&q1=http%3A%2F%2Fwww.example.com%2F
96pages from example.com&q1=http%3A%2F%2Fwww.example.com%2F
89links to example.com from source.com&q1=http%3A%2F%2Fwww.example.com%2F&q2=http%3A%2F%2Fsource.com%2F
109links from source.com to example.com&q1=http%3A%2F%2Fsource.com%2F&q2=http%3A%2F%2Fwww.example.com%2F
3latest pages&o=12345
4latest URLs&o=12345
5latest edges&o=12345
16page content for http://www.example.com/&q1=http%3A%2F%2Fwww.example.com%2F
10crawl download links

Note that PR is meaningless (and not calculated) in 'seed only' crawls and "popular pages" actions are therefore useless in those cases. For regular 'unlimited' crawls however, these are very useful commands. Also, for single site crawls it is useful to analyze popular pages to see website's internal linking effects.

contact | terms | privacy
© 2016 semanticjuice.com