One port, N catalogs: importing products into a .NET ecommerce platform

TL;DR

• Every catalogue source in Tendero is one class behind one .NET interface, registered by name, with no if to grow.
• The real contract is an xUnit test suite that every new connector inherits before it can merge.
• Importing is idempotent, never publishes, and stores each image under the hash of its bytes.

Every commerce platform I have worked on ends up with the same folder. It is called Importers, or Integrations, or Feeds, and it holds one class per supplier: ShopifyImporter, PrestashopImporter, TheOneForTheSupplierWhoSendsCsvByEmail. Each one reads from somewhere and writes products. And each one has its own idea of what a product is.

The folder is not the problem. The problem is that nothing makes those ideas agree. Over time they move apart, and nobody sees it until a supplier’s data reaches a code path that only one importer has.

Tendero is a public project I am building to work through exactly these questions. It is an AI-native ecommerce platform: .NET 11 and Aspire on the backend, two Angular apps on the front, and everything self-hosted with Docker. The catalogue is where it starts, because a shop with no products is not a shop.

This is what I did instead of the folder.

One interface, and it streams

public interface ICatalogSourceConnector
{
    /// <summary>Stable lowercase identifier: "seed", "shopify", "medusa"...</summary>
    string Source { get; }
    IAsyncEnumerable<ExternalProduct> StreamProductsAsync(
        CancellationToken cancellationToken = default);
}

These six lines already contain two decisions.

The first is IAsyncEnumerable, not Task<IReadOnlyList<T>>. The dataset I am building towards is Amazon Berkeley Objects (ABO), with about 147,000 products. I chose it because it is a real catalogue, not an invented one: Amazon listings with titles, brands and product types in several languages, Spanish among them, and almost 400,000 product photos. Its licence, CC BY 4.0, allows commercial use, which this project requires of everything it depends on. It is big enough to show what a toy catalogue hides. A method that returns a list says «load the whole catalogue into memory first». Once that is in the interface, no amount of care later can take it back out. The seed connector reads its JSON with JsonSerializer.DeserializeAsyncEnumerable, so the file is never in memory all at once.

The second is that Source belongs to the connector. The caller does not pass it in, because the connector knows what it is. That looks like a small detail, but it decides how a second import recognises the products of the first one. I come back to it below.

The product that the connector returns is multilingual from the start:

public sealed record ExternalProduct(
    string ExternalId,
    IReadOnlyDictionary<string, string> Names,
    IReadOnlyDictionary<string, string>? Descriptions,
    string? Brand,
    string? Category,
    decimal PriceAmount,
    string PriceCurrency,
    IReadOnlyList<ExternalImage> Images,
    IReadOnlyDictionary<string, string> Attributes,
    DateOnly? AvailableFrom = null);

Texts arrive as pairs of culture and value. A source that only speaks one language sends one key, and a later step fills in the rest. The alternative is a Name string plus a Culture field. But then every consumer has to guess which language it just received.

Adding a source is a registration, not an if

Connectors register as keyed services, one per source name:

services.AddKeyedScoped<ICatalogSourceConnector, SeedCatalogConnector>(
    SeedCatalogConnector.Key);

The name arrives in the request (POST /api/catalog/import { "source": "seed" }), so the code must find the connector at runtime. The obvious way is to inject IServiceProvider into the handler and ask it. I did that, and it was wrong. It took a review to see why: a handler that receives the whole container depends on everything and declares nothing, and you cannot test it without building a container. So the container moved behind a port of its own:

public interface ICatalogSourceRegistry
{
    IReadOnlyCollection<string> Sources { get; }
    ICatalogSourceConnector Get(string source);
}

That gave me something I did not expect. The registry can list what is registered, so the command validator can check that a source exists:

public ImportProductsValidator(ICatalogSourceRegistry sources)
{
    RuleFor(command => command.Source)
        .NotEmpty()
        .Must(source => sources.Sources.Contains(source, StringComparer.Ordinal))
        .WithMessage(command =>
            $"Unknown catalog source '{command.Source}'. " +
            $"Registered sources: {string.Join(", ", sources.Sources)}.");
}

Before that, the validator only checked the shape of the string, with ^[a-z0-9-]+$. That pattern accepted "shopify" when there was no Shopify connector. The error then came out of the DI container as a 500: a mistake by the caller, reported as a fault of the server. Now it is a 400 that names the sources that do exist. This project expects AI agents to call its API, so «here is what you could have asked for» is a real help, not a small extra.

The handler does not know which connector it got:

var connector = sources.Get(command.Source);
await foreach (var external in connector.StreamProductsAsync(cancellationToken))
{
    // ...
}

Adding Shopify means one class and one registration line. There is no switch, and there is no place where one could grow.

The contract is a test suite, not a comment

An interface promises method names. It says nothing about whether your ids are unique, whether your prices are negative, or whether your image path means anything outside your own folder.

So the port comes with an abstract xUnit class, and every adapter inherits it:

public abstract class CatalogSourceConnectorContractTests
{
    protected abstract ICatalogSourceConnector CreateConnector();
    protected virtual int MinimumExpectedProducts => 1;
    [Fact]
    public async Task External_ids_are_unique_and_non_empty() { /* ... */ }
    [Fact]
    public async Task Every_product_has_localized_name_and_valid_price() { /* ... */ }
    [Fact]
    public async Task Image_locations_are_absolute_and_readable() { /* ... */ }
    // ...and two more: the source key is stable and lowercase, and the
    // connector streams at least MinimumExpectedProducts.
}

The concrete subclass is three lines:

public sealed class SeedCatalogConnectorContractTests : CatalogSourceConnectorContractTests
{
    protected override ICatalogSourceConnector CreateConnector() =>
        new SeedCatalogConnector(Options.Create(
            new SeedConnectorOptions { FilePath = "TestData/products.sample.json" }));
    protected override int MinimumExpectedProducts => 6;
}

A new adapter without its subclass does not merge. That rule is the only thing that stops «N catalogs» from becoming «N dialects».

The most interesting test in that suite is the one about images. It does not require HTTP, because the seed connector reads from disk, and a later document-ingestion step will write to a temporary file. What it requires is that the location works without any context around it:

Assert.True(image.IsAbsoluteUri(),
    $"Relative image location in '{product.ExternalId}': {image.Location}.");

A relative path only works if the reader happens to be in the right folder. That stops being true as soon as something runs from somewhere else. For me, it happened the first time I started a fresh clone of the repository the proper way, from the Aspire AppHost. The seed file was looked for in the API’s own folder, where it does not exist, and the import answered 500. All my earlier tests used an absolute path, so none of them could see it.

Importing never publishes

The connector produces an ExternalProduct. Something has to turn it into the Product aggregate, and that mapping is where an important decision is made without anyone noticing:

public static Product ToNewProduct(
    this ExternalProduct external, string source, TimeProvider clock,
    AttributeDefinitions? definitions = null)
{
    var product = Product.Create(
        clock,
        external.LocalizedName,
        external.Price,
        external.LocalizedDescription,
        external.Brand,
        external.Category);
    product.SetAvailableFrom(clock, external.AvailableFrom);
    product.LinkExternal(clock, source, external.ExternalId);
    CopyAttributes(external, product, clock, definitions);
    EnsureDefaultVariant(external, product, clock);
    return product;
}

The clock is an injected TimeProvider, so a test decides what time it is instead of asking the machine. EnsureDefaultVariant gives every product one variant, even when its source knows of none, so everything you can buy has the same shape.

Product.Create puts the product in Draft, and nothing here publishes it. Draft means «imported, waiting for review». Search only indexes Active products, and moving from one status to the other is an explicit operation that somebody has to run.

That decision has a real cost, and I paid all of it. It is the subject of a later article. Here I will only say that a catalogue nobody publishes is a catalogue nobody can find.

Idempotency lives in the same place. A second import finds each product by (source, externalId):

var existing = await repository.FindByExternalReferenceAsync(
    source, external.ExternalId, cancellationToken);

and the reference itself refuses to be added twice:

public void LinkExternal(TimeProvider clock, string source, string externalId)
{
    ArgumentException.ThrowIfNullOrWhiteSpace(source);
    ArgumentException.ThrowIfNullOrWhiteSpace(externalId);
    var reference = new ExternalReference(source.ToLowerInvariant(), externalId);
    if (!this.externalReferences.Contains(reference))
    {
        this.externalReferences.Add(reference);
        Touch(clock);
    }
}

This is where Source on the connector is useful. The handler links every product to connector.Source, the name the connector gives itself, and never to the string from the request. The code that knows where a product came from is the code that decides what it is linked to.

Running the import twice changes nothing and costs one query per product. Running it after a supplier changed their prices updates the prices. That is the whole contract. It matters because import is exactly the operation people repeat when they are not sure it worked.

Images are stored, not linked

The easy option is to store the supplier’s image URL. It is one string, it costs nothing, and it works.

It works until the supplier reorganises its servers. Then your catalogue is a grid of broken images, and none of the pictures are yours. So the import downloads the bytes and stores them itself:

var content = await imageReader.OpenAsync(image, cancellationToken);
if (content is null)
    continue;
await using var stream = content.Content;
var id = await imageStore.SaveAsync(stream, content.ContentType, cancellationToken);
product.AddImage(clock, id, image.LocalizedAlt);

Two things in that code are deliberate.

A failed image does not fail the product. OpenAsync returns null for a 404, a dead server or a file that moved. An import of 147,000 products must not stop at image number 3,000. The product goes in without the photo, and the next import tries again.

The image id is the hash of its content, not a GUID:

public readonly record struct ImageId(string Value)
{
    public override string ToString() => Value;
}

Two products with the same photo get the same id, and the file is written once, so you get deduplication for free. Even better, an id can never point to different bytes than yesterday. So the endpoint that serves it can say Cache-Control: public, max-age=31536000, immutable, and it is true. A new photo for a product does not invalidate a cache. It simply gets a different id.

That store had a small race condition, and it is worth showing, because nobody writes a test for this kind of bug. The first version checked whether the destination file existed, and then moved the temporary file onto it. Two imports of the same photo could both pass the check, and the second one threw an IOException. The fix is to stop checking and let the move fail:

try
{
    File.Move(staging, destination, overwrite: false);
}
catch (IOException) when (File.Exists(destination))
{
    logger.LogDebug("Image {ImageId} already stored", id);
}
finally
{
    if (File.Exists(staging))
        File.Delete(staging);
}

It does not matter which import wins. The bytes are identical, because the id is the hash of the content. That idea helps a second time here.

The part I got wrong for a whole afternoon

I wanted sample images for the seed catalogue, with licences I could check. Openverse and Wikimedia both give licence data through their APIs, so the legal part took one morning.

The useful part never happened. A search for «running shoes» under CC0 returned a pair of trainers hanging from a power line. Only two of six results were usable. Searching for «isolated white background» left almost nothing, and allowing CC BY as well as CC0 barely helped. Free image banks have photos of things in context, not product photos. Product photography is specific, boring, commercial work, and nobody uploads it for free.

In the end the seed’s pictures are simple illustrations of my own, in the colours of the design system. Four attempts at a realistic shoe failed so badly that I have to admit it.

The idea that solved it was better than the images, and someone had to say it clearly before I saw it: the images in the repository are not Tendero’s images. They belong to the fake supplier. The seed connector simulates an external catalogue, exactly like Shopify would. A real source keeps its photos on its own servers. This one keeps them in seed/images/, and Tendero imports both in the same way. Nobody complains that product names in a JSON file are «not how real ecommerce works», because that JSON is what the shop imports from. It is not the shop.

What the dispatcher taught me about my own rules

One more story, because this kind of problem only appears when you finally test the code you trusted most.

Tendero uses its own CQRS abstractions instead of MediatR. The project’s licensing policy requires every dependency to have a permissive licence for commercial use, and several .NET staples moved to commercial licences in 2025 and 2026. So the dispatcher is my own code, and everything else in the repository depends on it.

It had no tests. I wrote eight. They failed at once and found two real bugs.

Handler registration used GetExportedTypes(), which only returns public types. An internal handler was never registered, without any warning, and failed at runtime with «No handler registered». It only worked because every handler so far happened to be public. But the architecture rules push the other way, because adapters should be internal. It uses GetTypes() now, with a comment that explains why.

Handlers were called through MethodInfo.Invoke, so any exception a handler threw reached the caller wrapped in a TargetInvocationException. An endpoint could never catch the domain exception it was written for. Now the dispatcher builds a closed generic wrapper once per request type and calls the handler through the interface. Reflection happens once, and the exception arrives unwrapped.

Neither bug is unusual. Both were invisible until the day I wrote a test to look for them.

What it cost, and what it gave back

The port, the product type, the registry, the contract suite and the mapper are about 380 lines, comments included. That is more than a single SeedImporter class would have been. If Tendero only ever read one JSON file, it would be over-engineering.

It is not, and the test suite is the reason I can say so honestly. The contract runs as code. When the Shopify adapter arrives, I will not wonder whether it behaves like the seed one. Five inherited tests will tell me, before I write any of the import logic.

With AI writing the code, tests are worth more

This matters more now that an AI agent writes much of the code. An agent can write a new connector in minutes, and it will look right. Whether it behaves like the others is a different question, and reading the code is a slow way to answer it. The suite answers it in seconds, every time, for every connector, whoever or whatever wrote it. When code becomes cheap to write, the tests are the part that holds what you actually decided.

The same agent changes what tests cost. Writing them used to be the slow, boring part, and the first thing a deadline cut. Now an agent writes a test as fast as it writes the code, so «no time for tests» stops being an excuse. Three kinds of test do different work here:

• Unit tests check one piece of logic in milliseconds: the mapper, the dispatcher, the connector’s contract. They run on every change, so a mistake the agent makes shows up in seconds, not in a review.
• Integration tests run against a real PostgreSQL in a container, with Testcontainers. They catch what an in-memory fake cannot: a mapping the database refuses, or a migration that builds a different schema from the one the code expects. The relative image path above is the same kind of bug. It only appears when the real pieces run together.
• Mutation tests check the tests themselves. You break one line of the code on purpose, for example GetTypes() back to GetExportedTypes(), and a good test must fail. A test an agent wrote can pass for the wrong reason, and this is how you find out. In the code of this article I make each mutation by hand. Stryker.NET, which makes them automatically across a whole project, comes in a later phase.

So the agent writes the code fast, the tests decide whether it is right, and the mutations decide whether the tests can tell.

Summary and conclusions

• One port for every source. A new catalogue is one class and one registration line, never a new if.
• The contract is a test suite. Every adapter inherits it, and an adapter without it does not merge.
• Importing never publishes. A product arrives as a draft, and a second import changes nothing.
• Store images, do not link them. The id is the hash of the bytes, so duplicates disappear and caching is safe.
• Test the code you trust most. Eight tests for the dispatcher found two bugs that nothing else had seen.
• With AI writing the code, tests are worth more. Unit, integration and mutation tests now cost little to write, and they are what checks code that cost little to write.

The repository is public and MIT licensed, and the code exactly as this article describes it is tagged blog/02-one-port-n-catalogs. To run it, clone the repository and start it with dotnet run --project src/AppHost.

How we know it works

• The test that fails: make the seed connector return a relative image path, and Image_locations_are_absolute_and_readable fails. Register handlers with GetExportedTypes() again, and the dispatcher’s own tests fail with «No handler registered».
• The check on every pull request: the build treats warnings as errors, and it runs the architecture rules and the 547 backend tests, including the connector contract suite.
• The mutation score: not yet. Stryker.NET comes in a later phase, and this line will be filled in from the first article whose tag includes it.

An interface promises names. A test suite promises behaviour.

Next article: Search with regression tests: NDCG as a CI gate — having products is not the same as being able to find them, and measuring the difference is where the interesting failures were.

Tendero — commerce for humans and agents.