Indexing
The states a document goes through before it can answer, and what to do when one fails.
Between uploading a document and seeing it in an answer there is processing in stages. Each document carries the state of the stage it is in.
Only Active documents take part in answers. Every other state is intermediate or an error.
The states
| State | What is happening |
|---|---|
Queued | The file has arrived and is waiting to be processed. |
Extracting | The content is being turned into text. |
Indexing | The text is being split into passages and prepared for search. |
Active | The document is available for answers. |
Extraction failed | No text could be obtained from the file. |
Indexing failed | The text was obtained, but didn't make it into the search index. |
The total time depends on the file's size and type. Scanned PDFs take longer because extraction has to recognize the text in the image.
Neither error state recovers on its own. A document stuck in one stays invisible to Agents indefinitely, with no warning on the Agent screen.
When extraction fails
Extraction failed almost always means the file has no recoverable text:
- A PDF that is only the image of a scanned page, with no text layer.
- A corrupted or password-protected file.
- A format the platform doesn't process.
The fix is to convert the file to a format with selectable text and upload it again. Re-uploading the same file produces the same failure.
When indexing fails
Indexing failed is rarer and doesn't depend on the format: the text was obtained, but didn't make it into the index.
Remove the document and upload it again. If the failure repeats with the same file, the content has some quirk worth reporting to support, with the document at hand.
The effect of a stuck document
An Agent attached to a Collection whose documents are still processing answers as if the Collection were empty. There is no warning, in Admin or in Chat.
So when you publish an Agent right after creating the Collection, come back to this tab and confirm that the documents reached Active before telling people.
Next steps
- Manage the documents in Files.
- Understand how indexed passages become an answer in Collections and knowledge.