Over the last few weeks I've been working on integrating with a third-party electronic health records integration API. The low-fidelity summary is as follows:
- Accept an event as a JSON payload in an HTTP POST request
- Validate the event payload
- Deserialize into a Python object
- Perform business logic with that object
Because I want the deserialized events to have discoverable, typed attributes, I started with a simple
class SampleEvent: def __init__(self, json_dict): self._json = json_dict def first_name(self) -> str: return self._json["FirstLevel"]["SecondLevel"]["FirstName"]
Not bad. Then I encountered fields that might be present. Consider the scenario in which
$.FirstLevel.SecondLevel.FirstName in the data model for this integration API is optionally present. Here's one implementation:
class SampleEvent: def __init__(self, json_dict): self._json = json_dict def first_name(self) -> Optional[str]: return self._json["FirstLevel"]["SecondLevel"].get("FirstName")
Cool. And what if (as I encountered), one of the ancestors of
FirstName is optional? Rather than chain a bunch of calls to
getattr or have a massive if-block, we could do this:
class SampleEvent: def __init__(self, json_dict): self._json = json_dict def first_name(self) -> Optional[str]: try: return self._json["FirstLevel"]["SecondLevel"]["FirstName"] except (KeyError, TypeError): return None
It's not...the worst code I've seen. However, even if I can accept its verbosity, we have scores of properties like this. In addition, the mental mapping between our data provider's documentation using JSON paths (e.g.
$.FirstLevel.SecondLevel.FirstName) and our Python implementation's arrays (e.g.
self._json["FirstLevel"]["SecondLevel"]["FirstName"]) uses a cognitive cycle that I'd rather not expend.
Ideally, what I'd have is an implementation that does the following:
- Expresses property origins as JSON paths
- Exposes typed properties for discoverability by other engineers, including my future self
- Allows for parsing into custom types
- Avoids duplicated complexity/verbosity with each property addition
import jsonpath_ng from datetime import date, datetime from typing import Optional, Any, Dict, Callable def json_property(json_path: str, converter: Optional[Callable] = str): def _json_attribute(value_dict: Dict, path: str, converter: Callable) -> Optional[Any]: """ Gets a value from the provided dict using the provided json_path. If a value is found, the provided converter function will be used to generate a value. If no value is found, the return value is None. """ jsonpath_expr = jsonpath_ng.parse(path) try: value = jsonpath_expr.find(value_dict).value if value is None: # Don't convert None to "None" return value return converter(value) except IndexError: return None return property( lambda self: _json_attribute( self._json, json_path, converter ) ) def _date_from_event_string(date_str: str) -> date: return datetime.strptime(date_str, "%Y-%m-%d").date() class SampleEvent: first_name: Optional[str] = json_property("$.FirstLevel.SecondLevel.FirstName") dob: Optional[date] = json_property("$.FirstLevel.SecondLevel.DOB", _date_from_event_string) def __init__(self, json_dict: Dict): self._json = json_dict