API.Planning.Domains

An interface to the repository of PDDL domains and problems.

Description

The planning.domains API provides programmatic access to a wide collection of PDDL benchmark domain and problem files. All of the physical files can be found in the repository of domains over at BitBucket. You can get a copy for yourself by cloning the repository, or using the command-line utility described below.

If you find any bugs in the PDDL files (incomplete sets, bad formulations, etc), clone the repository and submit your fix as a pull request. Where it makes sense, we will also accept pull requests with entirely new benchmark sets (e.g., after new IPC contests or as a release of a particular suite). For now, only classical PDDL benchmarks will be accepted (of any PDDL level / expressivity), but in the future we hope to expand this to other richer formalisms (POND, FOND, RDDL, etc).

The collection of files is just the first component of api.planning.domains. The primary aspect of this service is the interface offered to query, view, and explore the existing domains. We store, and continually update, attributes for each of the domains and problems. For example, the lower and upper bounds on plan cost, the requirements found in a domains, the classical width of a problem, etc. On this page, you can find a description of the interface to api.planning.domains, and details on the tools / libraries constructed to make interacting with the API even easier.

The API contains three types of objects:

  1. Problems: Information for each individual problem, including its file, the corresponding domain file, statistics on the problem, etc.
  2. Domains: Information for each individual domain, including its description and various statistics for the domain.
  3. Collections: Sets of domains, including a collection for every IPC, individual planner collections, etc.

Disclaimer: The API should be considered to be in a very alpha phase at the moment. The hope is to eventually have a way for citing precisely the benchmarks used in evaluating your software, but for the time being we are still fixing up the domains, replacing problem sets, and correcting bugs in the various statistics. Until this disclaimer is removed, we do not recommend using the API for official business (academic or otherwise).

Thanks: The initial statistics in the database for the API were due to the efforts of Miquel Ramirez, Patrik Haslum, and Nir Lipovetzky. We are working on a wiki-like interface for anyone in the community to contribute if they wish. If you would like to see a new type of statistic, or would like to help automatically generate existing ones, let me know or create an issue on our bug tracker.

API Interface

Responses and Formats

The default response for the API is JSON (application/json). The response will be wrapped in the following generic JSON object:


  {
    "error": False,
    "message": "Success!",
    "result": ...
  }
                    

The message value is usually useless unless error is set to True. The result value corresponds to the specific query, and we describe each in turn below.

The general path for the API is the following:

GET api.planning.domains/:format/classical

We will use API_PATH to refer to this path throughout this page. :format can be either json or xml depending on the return format you are interested in. Finally, classical is used to indicate that you wish to interact with classical planning problems and domains (other types will be added in the future).

Collections

Attribute Description
collection_idID of the collection.
collection_nameCollection name.
descriptionCollection description.
domain_setThe list of domain IDs in the collection.
tagsSet of tags associated with the collection (e.g., if the collection corresponds to an IPC).
GET API_PATH/collections

Returns all of the collections.

GET API_PATH/collection/{col-id}

Returns the collection matching col-id.

GET API_PATH /collections/search

Returns all of the collections matching the query provided. The only possible parameter for the moment is collection_name, and the collections returned will all match the string passed in.

POST API_PATH /updatecollection/{col-id}

Updates an attribute value for the collection col-id. Requires token authorization, and the following POST parameters are mandatory: (user) email address of the user; (password) token provided by the administrators; (key) name of the attribute to update; (value) new value of the attribute; (desc) short description of the change.

Domains

Attribute Description
domain_idID of the domain.
domain_nameDomain name.
descriptionDomain description.
tagsTags associated with the domain (e.g., the requirements such as ':strips').
GET API_PATH/domains

Returns all of the domains.

GET API_PATH/domains/{col-id}

Returns all of the domains in the collection corresponding to col-id.

GET API_PATH/domain/{dom-id}

Returns the domain matching dom-id.

GET API_PATH /domains/search

Returns all of the domains matching the query provided. The only possible parameter for the moment is domain_name, and the domains returned will all match the string passed in.

POST API_PATH /updatedomain/{dom-id}

Updates an attribute value for the domain dom-id. Requires token authorization, and the following POST parameters are mandatory: (user) email address of the user; (password) token provided by the administrators; (key) name of the attribute to update; (value) new value of the attribute; (desc) short description of the change.

Plans

GET API_PATH /plan/{prob-id}

Returns an IPC-style plan (i.e., one action per line) for the problem with id prob-id.

POST API_PATH /submitplan/{prob-id}

Allows the user to submit an IPC-style plan (i.e., one action per line) for the problem with id prob-id. It will overwrite the existing one if the plan validates and has a cheaper cost. Mandatory POST parameters include plan and email.

Tags

GET API_PATH /tags

Returns a list of all available tags, which includes their name, id, and description.

GET API_PATH /tag/{tag-id}

Returns info on the particular tag matching tag-id.

POST API_PATH /addtag

Adds a new tag. Requires token authorization, and the following POST parameters are mandatory: (user) email address of the user; (password) token provided by the administrators; (name) name for the new tag; (description) short description of the tag.

POST API_PATH /updatetag

Updates an existing tag. Requires token authorization, and uses the same POST parameters as adding a new tag.

POST API_PATH /tag[TYPE]/{item-id}

Tag a [TYPE](collection, domain, or problem) with the given item-id. Requires token authorization, and the following POST parameters are mandatory: (user) email address of the user; (password) token provided by the administrators; (tag_id) the ID of the desired tag.

POST API_PATH /untag[TYPE]/{item-id}

Used to remove a tag from a [TYPE](collection, domain, or problem) with the given item-id. Same POST parameters are required as the tagging functionality.

Problems

Attribute Description
problem_idID of the problem.
domain_idID of the domain this problem belongs to.
domainThe name of the domain this problem belongs to.
problemName of the problem (e.g., 'prob1.pddl')
domain_urlURL to the domain file.
problem_urlURL to the problem file.
domain_pathRelative path for the domain file.
problem_pathRelative path for the problem file.
upper_boundUpper bound on the plan cost (typically from a satisficing solution).
upper_bound_descriptionDescription of how the upper bound was computed (typically including the plan for verification).
lower_boundLower bound on the plan cost.
lower_bound_descriptionDescription of how the lower bound was computed (method, provenance, other weaker lower bounds, etc).
average_effective_widthThe average classical width of the problem over all goal fluents.
max_effective_widthThe maximum classical width of the problem over all goal fluents.
average_effective_width_descriptionDescription of the average effective width computation.
max_effective_width_descriptionDescription of the max effective width computation.
tagsList of tags for the problem.
GET API_PATH/problems

Returns all of the problems. Note: this may be disabled in the future, and it is not recommended for frequent use.

GET API_PATH/problems/{dom-id}

Returns all of the problems in the domain corresponding to dom-id.

GET API_PATH/problem/{prob-id}

Returns the problem matching prob-id.

GET API_PATH/nullattribute/{attribute}

Returns a list of info on the problems that have a null setting for attribute. Each item in the list contains a problem id, domain_path, and problem_path.

POST API_PATH /updateproblem/{prob-id}

Updates an attribute value for the problem prob-id. Requires token authorization, and the following POST parameters are mandatory: (user) email address of the user; (password) token provided by the administrators; (key) name of the attribute to update; (value) new value of the attribute; (desc) short description of the change.

GET API_PATH /problems/search

Returns all of the problems matching the query provided. The following parameters can be used for the query:

Param Value Description
domainNumberMatches the provided domain ID.
domain_nameStringMatches when the problem's domain name contains the provided string.
problem_nameStringMatches when the problem's name contains the provided string.
effective_widthNumberMatches all problems with the provided effective width.
average_effective_widthNumberMatches all problems with an average effective width equal to the provided number.
max_effective_widthNumberMatches all problems with a maximum effective width equal to the provided number.
min_lower_boundNumberMatches all problems with a lower bound no smaller than the provided number.
max_lower_boundNumberMatches all problems with a lower bound no larger than the provided number.
min_upper_boundNumberMatches all problems with an upper bound no smaller than the provided number.
max_upper_boundNumberMatches all problems with an upper bound no larger than the provided number.
openbounds[0/1]Matches problems that have an open lower and upper bound (when 1 is used as a value) or a closed lower and upper bound (when 0 is used as a value).

As an example query, the following finds all problems with an effective width of 1 (so they are in essence "easy" to find a solution for), and also have open bounds (so they are "hard" to prove optimality for):


Note: Until the statistics are improved on, many problems will not have any effective width value, and so the above query may not return much.

API Tools

Purpose

To demonstrate what is possible with the API, and to provide a more practical interface to the data, we have created a set of tools which can be found in the api-tools repository.

Python Library

The planning_domains_api.py library provides access to the API through the following functions (more to be added in the future):

Method Description
query(qs)Query for the string http://api.planning.domains/json/classical/{qs}
simple_query(qs)Helper function that calls query(qs) and wraps it with a bit of error handling. This is recommended over query(qs).
get_collections()Return the list of collections.
get_collection(cid)Get the collection object corresponding to the collection id cid.
find_collections(name)Return the list of collections with a name matching the passed in argument.
get_domains(cid)Get the list of domain objects that are in collection cid.
get_domain(did)Get the domain object corresponding to the domain id did.
find_domains(name)Return the list of domains with a name matching the passed in argument.
get_problems(did)Get the list of problem objects that are in domain did.
get_problem(pid)Get the problem object corresponding to the problem id pid.
find_problems(name)Return the list of problems with a name matching the passed in argument.
localize(prob)Changes the file paths in a problem object to the appropriate local path assuming you have set it using the planning.domains.py utility.
get_tags()Get the list of all tags.
tag_TYPE(id,tname)Update a TYPE(collection, domain, or problem) with a given id and tagname.
untag_TYPE(id,tname)Remove the tag from a TYPE(collection, domain, or problem) with a given id and tagname.
update_TYPE_stat(...)Updates an API attribute for a given TYPE(collection, domain, or problem) given the following parameters: (iid) item id; (attribute) name of the attribute; (value) new value; (description) info about the update. An additional helper function exists to find all problems with a missing attribute: get_null_attribute_problems(attribute)
get_plan(pid)Get the IPC-style plan for problem pid.
submit_plan(pid,plan)Submit a new IPC-style plan for problem pid.

Example that demonstrates how to test on every problem in the "All IPC" set (note, this assumes that you have already run the command line utility to set up the location of the planning. domains repository):

import sys

print "Loading domains...",
sys.stdout.flush()

import planning_domains_api as api

# 12 is the collection for all STRIPS IPC domains
domains = {}
for dom in api.get_domains(12):

    # Turn the links into relative paths for this machine
    probs = map(api.localize, api.get_problems(dom['domain_id']))

    # Map the domain name to the list of domain-problem pairs
    domains[dom['domain_name']] = []
    for p in probs:
        domains[dom['domain_name']].append((p['domain_path'], p['problem_path']))

print "done!"

Command-line Utility

The planning.domains.py command-line utility allows you to setup your repository of domains and do primitive queries. It will check out the domains repository if you wish, and store a local settings file so that it can be automatically retrieved using the planning_domains_api.py library.

The local settings file is also used by the API in order to store the user's email and (if available) access token for making changes to the API attributes.

The tool is interactive to set things up, so just run ./planning.domains.py from the command line to get started. You can also run it without parameters to get the usage:

> ./planning.domains.py

 No command-line options given.  Usage:

 planning.domains.py update                                Update the local domain repository.

 planning.domains.py register                              Register your email and token for making API edits

 planning.domains.py find collections [string]             Find collections whose title/ID contains 'string'
 planning.domains.py find domains [string]                 Find domains whose title/ID contains 'string'
 planning.domains.py find problems [string]                Find problems whose title/ID contains 'string'

 planning.domains.py show collection [integer]             Find collections whose title/ID contains 'integer'
 planning.domains.py show domain [integer]                 Find domains whose title/ID contains 'integer'
 planning.domains.py show problem [integer]                Find problems whose title/ID contains 'integer'
 planning.domains.py show plan [integer]                   Show the plan (if any) matching the given problem ID

 planning.domains.py list collections                      Lists all of the collections.
 planning.domains.py list tags                             Lists all of the available tags.
 planning.domains.py list null-attribute [string]          Lists all of the problems that have a null attribute setting (string)

 planning.domains.py tag collection [integer] [string]     Tags the specified collection (integer) with a tag (string)
 planning.domains.py tag domain [integer] [string]         Tags the specified domain (integer) with a tag (string)
 planning.domains.py tag problem [integer] [string]        Tags the specified problem (integer) with a tag (string)
 planning.domains.py untag collection [integer] [string]   Un-tags the specified collection (integer) with a tag (string)
 planning.domains.py untag domain [integer] [string]       Un-tags the specified domain (integer) with a tag (string)
 planning.domains.py untag problem [integer] [string]      Un-tags the specified problem (integer) with a tag (string)

 planning.domains.py submit plan [integer] [plan file]     Submit the provided plan for validation and possible storage

 planning.domains.py generate-lab [integer] [string]       Generate a lab-style python file (string) given a collection (integer)
                    

Javascript Library

The planning-domains.js library makes it quick and easy to display problems, domains, and collections on a website. Both jQuery and Bootstrap are prerequisites, but other than that all you need to do is include the JavaScript file somewhere on your page:

<script type="text/javascript" src="planning-domains.js"></script>

Once loaded, you can call any of the following JavaScript functions to populate a div with the appropriate HTML:

  • insert_navigator(parent_div, selection_func_name)

    Inserts a full navigator of the collections / domains / problems in the API as the HTML text of parent_div. The function matching the name selection_func_name will be invoked with a single parameter corresponding to the problem ID once the user has navigated to a problem and selected it. The selection_func_name parameter is optional, and to see it in action, use the "Import" functionality of editor.planning.domains.

  • fetch_collections(query_string, parent_div, selection_func_name)

    Populates parent_div with a table of collections matching the query string query_string (e.g., '/collections'). The option selection_func_name will be used as a callback if a row corresponding to a collection is selected, and the collection ID will be used as a parameter.

  • fetch_domains(query_string, parent_div, selection_func_name)

    Populates parent_div with a table of domains matching the query string query_string (e.g., '/domains/12'). The option selection_func_name will be used as a callback if a row corresponding to a domain is selected, and the domain ID will be used as a parameter.

  • fetch_problems(query_string, parent_div, selection_func_name)

    Populates parent_div with a table of problems matching the query string query_string (e.g., '/problems/search?domain_name=blocks'). The option selection_func_name will be used as a callback if a row corresponding to a problem is selected, and the problem ID will be used as a parameter.

For examples, see the /web directory of the API Tools Repository.