AI is a powerful technology. I use it myself, and I have spent a lot of time testing different AI models while building TariffWolf.
But there is a problem when you put a general-purpose AI model/LLM at the center of a trade compliance classification system: HS classification needs predictable results. An LLM does not always work that way.
When I started building our automated tariff classification engine, I used public LLMs heavily. I initially thought the main challenge was choosing the right model. I moved between OpenAI, Gemini and Anthropic and tested the results against different products.
The accuracy improved from model to model, but it never became reliable enough for the core classification logic.
That was the point where I started looking at the problem differently.
The issue was not simply the model. The issue was the way an LLM reaches an answer.
An LLM is very good at finding a likely answer based on the information it has seen. HS classification, however, often requires something more specific. The answer has to follow the tariff version applicable on a particular date, the General Rules for Interpretation, chapter and section notes, exclusion notes, product characteristics and, in some cases, customs rulings.
A classification engine cannot simply return the answer that looks most likely.
It needs to return the answer that follows the applicable rules.
The tariff version problem
One of the things that concerned me during testing was how an LLM deals with changes between tariff versions.
For example, in my testing, the classification of a smartphone changed between tariff versions. In the 2017 tariff version, “Smart Phone” was classified under subheading 8517.12, while in the 2022 tariff version it was classified under 8517.13.
Now imagine asking an AI model to classify a smartphone imported on 1 January 2022.
The model may know the newer classification and return 8517.13. But that does not necessarily mean it has established which tariff version and legal provisions apply to the particular transaction.
This matters. A lot.
A classification system should not answer a historical classification question using whichever version of the tariff happens to be most prominent in its training data.
This was one of the reasons I became uncomfortable with putting an LLM directly inside the core classification process.
The sugar-free chewing gum test
I have one classification test that I have used repeatedly while testing AI classification tools: “sugar-free chewing gum”.
In my testing, several AI classification tools returned 1704.10.
At first glance, the answer looks reasonable.
Subheading 1704.10 describes chewing gum, whether or not sugar-coated. An LLM sees the words “chewing gum” and “sugar” in the product description and finds an obvious match.
But the problem is that tariff classification does not work by keyword matching.
The product being tested was “sugar-free chewing gum”, and the classification has to be considered in the context of the relevant chapter provisions and exclusions. The classification referenced in my testing was 2106.90, and CBP material also contains examples of sugar-free or synthetic-sweetener confectionery being classified within heading 2106. Here’s what Claude did:
The interesting part is not just that it returned the wrong code.
The interesting part is why it returned the wrong code.
The answer appears to follow the strongest textual match:
“chewing gum” → 1704.10
That is understandable for a language model.
But an HS classification engine needs to ask a different question:
Does the product actually satisfy the legal requirements for that heading and subheading?
Those are two very different approaches.
The screenshot shows how easily a plausible explanation can be produced around an incorrect classification. The explanation sounds confident because the words in the product description match the wording of the tariff heading.
That is exactly what makes this type of error dangerous.
The T-shirt example
The second test was an innocuous T-shirt.
When I asked an AI classifier about a 100% cotton T-shirt, it returned 6109.10.
For a normal knitted cotton T-shirt, that answer can look perfectly reasonable. Heading 6109 covers T-shirts, singlets and similar garments, and 6109.10 covers those of cotton.
The problem appears when you look at the actual product image. Let’s try Claude again:
The garment in the test image has a mock or turtleneck collar.
That detail changes the classification analysis.
The CBP material referenced in my original testing explains that garments meeting the basic T-shirt description can be excluded from heading 6109 when they have a conspicuous non-T-shirt feature. Examples include a mock turtleneck or high crew neckline, among other features. Such garments can fall under heading 6110.
So in this case, simply identifying the garment as a cotton T-shirt is not enough.
The model looked at the overall appearance and the familiar product description and returned 6109.10. It did not ask the additional classification questions that a human classifier would normally consider.
That is the part that matters to me.
The missing follow-up questions
Another pattern I noticed during testing was that the AI classifier often did not ask follow-up questions.
This is a serious issue for automated classification.
Take a simple product description such as:
“Key made of steel and plastic.”
There are several questions that may matter before assigning a classification.
What type of key is it?
Is it presented separately?
Is it a key for a lock or another device?
What is the role of the steel component?
What is the role of the plastic component?
Is the steel component providing the essential character?
Instead of asking these questions, Claude often went directly to an HS code.
In the example from my testing, the AI returned 8301.70 and provided a reasonable-sounding explanation based on keys of base metal.
But the problem is that the explanation comes after the model has already selected the answer.
A classification system should sometimes stop and say:
“I need more information before I can classify this product.”
That is not a failure of the system.
In trade compliance, asking for the missing information can be the correct result.
Why we changed our approach
After testing different models, I came to a simple conclusion.
I do not want the core of our HS classification engine to depend on an AI model guessing the most likely answer.
That does not mean AI has no place in trade compliance.
AI can be useful for extracting product information, understanding descriptions, finding relevant text, helping users formulate questions and assisting with research. There are many places where an LLM can save time.
But the final classification decision is different.
For the core classification logic, we need the system to follow defined rules and the tariff data applicable to the transaction.
That means the engine needs to know things such as:
- which tariff version applies;
- the relevant heading and subheading;
- section and chapter notes;
- exclusions;
- product characteristics;
- the General Rules for Interpretation;
- and relevant customs rulings where applicable.
The system should be able to explain why it selected a code, and just as importantly, it should be able to explain why another apparently obvious code does not apply.
That is a very different architecture from simply asking an LLM:
“What is the HS code for this product?”
AI can still be part of the system
Removing AI from the core classification engine does not mean removing AI from TariffWolf.
I still believe AI has an important role to play.
The difference is where we put it.
If AI is used to help understand a product description, that can be useful.
If AI is used to identify information that may be missing, that can be useful.
If AI helps a user search through a large amount of tariff or customs information, that can also be useful.
But when the system reaches the actual classification decision, I want the result to come from a process that can be traced back to the applicable tariff rules.
That is the standard I want for our classification engine.
The lesson from building TariffWolf
When I started this project, I thought the hardest part would be finding the best AI model.
After working with the different models, I changed my mind.
The harder problem is designing a system that understands what kind of decision it is making.
HS classification is not a normal question-and-answer problem. A product can look like it belongs in one subheading because of a few keywords, while a note, exclusion, product characteristic or customs ruling points somewhere else.
An AI model can give you a very convincing answer in that situation.
That is exactly why we decided not to let the AI make the final decision.
For TariffWolf, the goal is not to produce the answer that sounds the best.
The goal is to produce a classification that can be traced, checked and defended.
That is why we removed AI from the core of our HS classification engine.