Chronicles of the Substack API: A Developer’s Guide to Tame
Chronicles of the Substack API: A Developer’s Guide to Taming the Beast
What starts as a simple quest—`‘let’s fetch some posts’`—can quickly spiral into an epic adventure. Our mission was to integrate a Substack publication into the ChipsXP blog, a task that led us through a labyrinth of cryptic errors, phantom data, and puzzling data structures. This chronicle details that journey, serving as a guide for fellow engineers venturing into the wilds of a new API.
Chapter 1: The First Encounter & The 500 Error Wall
Every developer knows the feeling: you write the code exactly as you think it should be, and the server responds with a dreaded `500 Internal Server Error`. Our first attempt was no different.
The Initial Mistake:
Our initial approach to building the API URL was based on a common REST pattern:```javascript
// The wrong path
const SUBSTACK_PUBLICATION = “jimmyburns.substack.com/”;
const endpoint = “posts/latest”;
// This looked right, but was fundamentally flawed.
const apiUrl = `https://substackapi.dev/api/v1/publications/${SUBSTACK_PUBLICATION}/${endpoint}`;
```
This consistently resulted in 500 errors. While the API playground worked, our code continued to fail. Our investigation revealed two critical misunderstandings:
The Base URL: The correct base URL was `https://api.substackapi.dev/`, not `https://substackapi.dev/api/v1/`.
The Parameters: The publication identifier was not a path parameter but a URL-encoded query parameter.
The Correct Incantation:
After consulting the (slightly confusing) sacred scrolls of API documentation, we found the correct spell:```javascript
// The right path
const SUBSTACK_PUBLICATION = “jimmyburns.substack.com/”; // From .env
const encodedPublication = encodeURIComponent(SUBSTACK_PUBLICATION);
const apiUrl = `https://api.substackapi.dev/posts/latest?limit=10&publication_url=${encodedPublication}`;
```
Lesson: An API playground is a great starting point, but always **double-check the exact URL construction**, including the domain and how parameters are passed.
Chapter 2: The Phantom Data Mystery
With the 500 errors vanquished, we were greeted with the holy grail: a `200 OK` response! However, the celebration was short-lived. The response was successful, yet our application insisted there were no posts.
This is where the developer’s most trusted tool comes into play: `console.log`.
The Investigation:
A simple log of the raw API response revealed the truth:```javascript
console.log(”Raw API response:”, JSON.stringify(apiData, null, 2));
```
The Smoking Gun:
The logged data showed a structure we had not anticipated:```json
{
“data”: [
{
“title”: “GENERATING A NOT TRASH PANDA 3D MODEL”,
“date”: “2025-09-09T18:01:46.772Z”,
“author”: “Jimmy Burns”,
“description”: “My Photogrammetry Workflow of a Garden Statue Raccoon”
}
// ... more posts
]
}
```
We were looking for a `posts` array, but the treasure was hidden inside a `data` array. It was a classic case of looking in the wrong pocket.
The Fix:```javascript
// Before: Looking for treasure in the wrong chest
const posts = apiData.posts; // This was undefined
// After: Found the map’s “X”
const posts = apiData.data; // This held our array of posts
```
Lesson: Never assume the shape of an API response, even if it seems obvious. Log the raw data to see the ground truth.
Chapter 3: The Great Translation Fiasco
Now that we had the posts, the next challenge was displaying them correctly. This led to a two-part mystery: mismatched field names and the case of the invisible images.
Our initial TypeScript interface was a well-intentioned guess:```typescript
// Our initial, incorrect interface
interface Post {
post_date: string;
canonical_url: string;
subtitle: string;
cover_image?: { url?: string };
}
```
The API, however, spoke a different dialect. The `console.log` from our previous investigation revealed the correct field names (`date`, `url`, `description`). However, the images were still a complete mystery. We assumed a simple `{ url: ‘...’ }` object, but no images appeared.
Another round of deep-logging the first post object was necessary.
The Image Structure Revelation:```json
// The smoking gun for images
First post cover_image: {
original: ‘https://substackcdn.com/image/fetch/...’,
og: ‘https://substackcdn.com/image/fetch/w_1200,h_630...’,
small: ‘https://substackcdn.com/image/fetch/w_150...’,
medium: ‘https://substackcdn.com/image/fetch/w_424...’,
large: ‘https://substackcdn.com/image/fetch/w_848...’
}
```
The API proved more sophisticated than we initially thought; it provided multiple, pre-sized versions of images. For our card-based layout, the `small` property was the perfect choice for performance and fit.
The Corrected, Bilingual Interface:
This discovery led to our final, battle-tested interface that accurately reflects the API’s structure:```typescript
interface Post {
title: string;
date: string;
author: string;
description: string;
url: string;
slug: string;
cover_image?: {
original?: string;
small?: string;
medium?: string;
large?: string;
};
author_image?: {
original?: string;
small?: string;
medium?: string;
large?: string;
};
// ... and other fields
}
```
Lesson: API objects are like onions; they have layers. Dig deep with logging to understand nested objects and avoid incorrect assumptions, especially for complex fields like images.
The Grand Technical Summary: The Final Working Code
After our trials and discoveries, we arrived at a robust and reliable solution.
1. Securely Fetching Posts:```javascript
// lib/substack.ts
// Secrets are loaded from environment variables, never hardcoded.
const SUBSTACK_API_KEY = import.meta.env.SUBSTACK_API_KEY;
const SUBSTACK_PUBLICATION = import.meta.env.SUBSTACK_PUBLICATION;
async function fetchSubstackPosts(endpoint: “posts/latest” | “posts/top”) {
const encodedPublication = encodeURIComponent(SUBSTACK_PUBLICATION);
const apiUrl = `https://api.substackapi.dev/${endpoint}?limit=10&publication_url=${encodedPublication}`;
const response = await fetch(apiUrl, {
method: “GET”,
headers: {
“X-API-Key”: SUBSTACK_API_KEY,
},
});
if (!response.ok) {
// Provide meaningful error messages
throw new Error(
`Failed to fetch ${endpoint}: ${response.status} ${response.statusText}`
);
}
const data = await response.json();
// Return the correct data array, not the whole object
return data.data || [];
}
```
2. Displaying Images Safely in a Component:
Using optional chaining (`?.`) is crucial to prevent errors when an image object or its properties are missing.```astro
// components/PostCard.astro
const { post } = Astro.props;
---
<div class=”card”>
{/* Use optional chaining (?. ) and select the ‘small’ property */}
{post.cover_image?.small && (
<img src={post.cover_image.small} alt={post.title} class=”cover-image” />
)}
<div class=”card-body”>
<h5 class=”card-title”>{post.title}</h5>
{/* Also for the author avatar */}
{post.author_image?.small && (
<img src={post.author_image.small} alt={post.author} class=”author-avatar” />
)}
</div>
</div>
```
Lessons Forged in the Fires of Debugging
APIs Are Picky Eaters: URL structure, query parameters, and encoding must be exact.
`console.log` is Your Infallible Oracle: When in doubt, log it out. It will reveal the true structure of the data you’re working with.
Assume Nothing: Do not assume field names or object structures. Verify everything.
Size Matters: When an API offers multiple image sizes, choose the one that best fits your design and performance goals.
Code Defensively: Use optional chaining (`?.`) and provide fallbacks to create a resilient UI that does not break when data is missing.
Keep Secrets Sacred: Never commit API keys or other secrets to version control. Use environment var!
