REST API Design Opinions: How Would You Allow Users to Request a Subset of Data?

Hypothetical Question for REST API Design Discussion:

Let’s say you’re building a credit reporting API that can return complete credit report profiles for an individual. The credit profile model is rather large and contains all the details for the customer. For example, personal details, addresses, aliases, risk models (credit score), tradelines (every bank account), etc…

I would probably make the API route look something like this:
(GET) /api/v1/creditProfiles/
The params would be the required information (name, ssn, address, etc), and the response object would contain a mementified CreditProfile object.

Now, what if the API consumer only cares about a subset of the data, like they only want the credit score/risk model instead of the whole massive CreditProfile?

Idea 1: New Nested Endpoint?
Would you create a new REST endpoint like this?
(GET) /api/v1/creditProfiles/riskModel/

Idea 2: Parameters to Control Data Returned
Or would you use the original (GET) /api/v1/creditProfiles/ endpoint but add a special boolean parameter like riskModelOnly=(true/false), which would return just the subset of data instead of the whole model? Similar to how we allow pagination or filter params to be passed in GET requests? I’ve seen a few .NET APIs work this way where you can customize what fields/data you want the endpoint to return.

Idea 3: Cruddy by Design
I’ve watched the “Cruddy by Design” video by Adam Wathan, which makes me think Adam would recommend creating a brand-new handler at the root of the API like this:
(GET) /api/v1/creditRiskModels/
My takeaway from Adam’s video is that he’s not a fan of building folder hierarchies and likes to stick with HTTP verbs as much as possible. In other words, if the action doesn’t match a verb, then create a new handler.

The “Cruddy by Design” approach smells off to me, though. I like the idea of handler names establishing a “domain” where users can easily predict what type of behavior, data, or actions are available by following a logical folder structure. For example, I know that /api/v1/creditProfiles will contain one or more endpoints (or nested endpoints) all related to the “creditProfiles” domain.

What do you think? How would you solve this type of conundrum, and why?

We approach this from the other direction: REST endpoints should return the smallest possible resource but allow for expandables, in the manner that Stripe uses them in their API:

This dovetails nicely with the Coldbox API handler and mementifier, because all it takes is a little bit of code in your API baseHandler:

// param rc.expand as an array for general-purpose REST field expansion
			event.paramValue( 'expand', [] );
			if ( !isArray( rc.expand ) ) {
				// if it's a string list, i.e. a GET param, convert it to an array
				if ( rc.expand.listLen( ',', false ) > 1 ) {
					rc.expand = rc.expand.listToArray( ',', false );
				}
				else {
					rc.expand = [ rc.expand ];
				}
			}

And then a reference in your endpoint handler when you want to support it:

var thing = wirebox.getInstnace( "someQuickEntity" ).withExpandables( rc.expand )

And then two functions in our custom Quick base entity that allow for invocation of expandables by mapping them to scopes on the object:

/**
	* getExpandedIncludes
	* @return An 'includes' mementifier argument that is the combination of the default
	*
	* @propertyName
	*/
	public array function getExpandedIncludes( array expand = [], array includes ) {
		// not all quick mementos will have an 'expandbale' property since we invented it, so default to an empty array if they don't
		var expandable = this.getDefaultMemento()?.expandable ?: [];

		// remove any expand values we asked for that aren't valid
		var validIncludes = arguments.expand.filter( ( item ) => {
			return expandable.findNoCase( item );
		} );

		// we don't want a default function argument that is calculated at compile time since this.memento isn't iniitialized until instanceReady(), but by the time we actually call this function the instance will be ready
		var initialIncludes = (
			arguments.keyExists( "includes" )
			 ? arguments.includes
			 : this.getDefaultMemento().defaultIncludes
		);

		return (
			len( validIncludes )
			 ? initialIncludes.append( validIncludes, true )
			 : initialIncludes
		);
	}

	/**
	* withExpandables
	* @return A (not-yet-loaded) Quick instance with scopes matching scopeWith{EXPANDABLE} triggered
	*/

	public function withExpandables( array expand = [] ) {
		var expandable = this.getDefaultMemento()?.expandable ?: [];

		// we might want to limit expandables to certain access levels; check for any contexts specific to each access level
		var expandableAuth = this.getDefaultMemento()?.expandableAuth ?: {};

		// remove any expand values we asked for that aren't valid
		var validIncludes = arguments.expand
			.filter( ( item ) => {
				return expandable.findNoCase( item );
			} )
			.filter( ( item ) => {
				return (
					!expandableAuth.keyExists( item ) || _wirebox
						.getInstance( "AuthService" )
						.authorize( context = expandableAuth[ '#item#' ] )
				);
			} );

		// remove any expand values that have auth contexts which we don't pass


		validIncludes.each( ( ex ) => {
			var expandoScope = 'scopeWith' & ex;
			if ( structKeyExists( this, '#expandoScope#' ) ) {
				this[ '#expandoScope#' ]( );
			}
		} );

		return this;
	}

This is a bit different than the question you asked, but as a general matter I think it’s counter-intuitive to have the ‘base resource’ be big and then add parameters to shrink it versus the other way around!

1 Like

This is awesome! What a clever idea!
I like the idea of starting small and allowing the user to “expand” the returned model with properties, relationships, etc.

You could probably also use memento profiles in a similar way where a user would specify the profile they want to get back from the mementifier if you wanted to give the users some presets (e.g. simple, compact, full, etc… )

I’m going to explore this idea in my own API design.

When you document the expand parameter, do you provide a list of available keys the user can provide? I don’t think Swagger files currently support this type of flexibility, but I don’t see why this information couldn’t live in the description field or as a few request/response examples.

Yes - we use the @x- extensions in swagger, like this:

	/**
	* /api/v1/family/:familyID
	* @hint View a family record
	* @param-familyID{36} { "schema" : { "type" : "string", "format" : "guid" }, "in" : "path", "description" : "Unique ID of the family to retrieve" }
	* @responses ~/family/get.responses.yml
	* @x-expandable [ "parent1", "parent2", "children", "familyParents" ]
	*/

I can’t remember if we had to change anything in cbswagger or redoc to do that, but if so, it was pretty straightforward. We do the same thing with @x-user-roles and @x-description.

1 Like