Support for Protocol Buffers as Request/Response Formats (BETA)¶
Valhalla allows users to interact with its service (and bindings/library) directly using pbf objects/bytes in addition to the conventional JSON request/response formats. This feature is currently in beta meaning that we are still actively developing it and there are likely still some bugs to work out before it's ready for widespread use. Most importantly, we reserve the right to make organizational changes to the protobuf schema while the label beta is still in effect.
Valhalla keeps track of the state of a "request" as it is being processed through our various APIs using protobuf messages. In fact, when you call into our top level APIs like
isochrones etc, we parse your json directly into a protobuf object. We then pass that object around to different parts of the library adding to it as we fulfill the request. We extended this functionality such that, instead of passing json to the service/bindings/library you can pass protobuf bytes or objects and get back protobuf bytes or objects.
There are a couple good reasons for this. As formats go, protobuf has many benefits over json:
- speed of serialization and deserialization
- over the wire size
- backward/forward compatibility
- working with an object on both sides of the network which eventually leads to
The major drawback is that protobuf is a binary format so if you have just the bytes they aren't much good to you.
To use protobuf as a request/input to the HTTP API you need to do two things:
- Send the proper HTTP header to signal a protobuf payload. The header should be:
- Send protocol buffer's serialized bytes as the body of the HTTP request
The message we use for the entire transaction is the
Api message, whose definition you can find here. All of the request parameters should be filled out via the
Options message attached to the
Api message. Most importantly, you will want to set your
pbf and your
action to the relevant API you are calling (though the HTTP request path also provides the latter). The
options object also contains a subobject named
pbf_field_selector which can be used to turn on/off the top level fields in the response. For example, if you only want the
directions part of the protobuf response to be present (much smaller payload) then turn on only that flag in the field selector. The rest of the request options depend on which API you are calling. For more information about what and which options to set for a given API please read that APIs specific docs regarding its request options.
As with the request/input, the response/output will again be the
Api message but will have more parts of it filled out. Depending on which API you are calling different parts of the response object will be filled out. Route-like responses will have
Directions objects filled out whereas non-route APIs will have different parts of the message filled out. Not all APIs support protobuf output. Those that don't, will return JSON as they do today. Currently, the following APIs support protobuf as output:
route, trace_route, optimized_route, centroid, trace_attributes, status
There are a few more things we should do before we can remove the beta label from this feature:
- Add Native PBF Support to Python Bindings: We can support, in addition to JSON strings, the ability for python to work directly with protobuf objects (those generated with protoc) across the python/c++ barrier. This would be a very natural way for python users to interact with Valhalla.
- Support for All APIs: As mentioned above we only support a certain subset Valhalla's APIs, over time we can add the rest of the APIs to the