What are Streams and How to Use Them in SortSpoke
TL;DR
- Streams are subsections within a project that let you organize different workflows while sharing the same document models and fields
- Use streams to separate testing/production environments, organize by team/client, or manage approval workflows
- Every project has a Default Stream automatically created
- Streams can be managed via the UI or API with specific endpoints for uploading and retrieving documents
- Access to streams can be controlled based on user roles and permissions
- Use streams when you want to maintain the same document structure; use projects when you need completely different models
What are Streams in SortSpoke?
Streams are subsections of a project used to organize work while maintaining a shared structure of models and fields. They function like "channels" within your project that allow you to:
- Separate different workflows within the same project
- Maintain access to the same project settings, including document models and extraction fields
- Organize documents and processing pipelines without creating entirely new projects
Every project has a Default Stream that's automatically created, and you can add custom streams as needed for your organization's workflow.
Why Use Streams?
Streams provide flexibility and organization to your SortSpoke workflows:
- Segregate testing and production environments to ensure quality before deployment
- Organize teams and workflows by assigning different streams for different groups
- Client segregation: Create separate streams for different clients while using the same document models
- Volume management: Distribute high-volume processing across multiple streams
- Approval workflows: Use different streams for different stages of approval
- Maintain project consistency while allowing flexibility in processing and analysis
Note: Streams share the same document models and extraction fields as the parent project. This makes them ideal for organizing workflows that use the same data structure but need to be separated for operational purposes.
How to Use Streams in SortSpoke
Creating and Managing Streams via UI
- Navigate to Project Settings
- Select the "Streams" tab
- Click "Create New Stream" and provide a name for your stream
- Assign team members to the stream as needed
- View stream-specific analytics from the dashboard by selecting the stream from the dropdown filter
Figure 1 - Admin > Project Settings
Figure 2 - Admin > Project Settings > Manage Stream
Figure 3 - Admin > Project Settings > Manage Stream > Add New Stream
Using Streams via API
1. Creating a Stream
Streams can be created within a project to manage separate workflows. Each stream is assigned a unique Stream ID.
2. Uploading Files to a Stream
Files can be uploaded directly to a stream rather than the default project:
POST api/v2/streams/{streamId}/upload
Example response:
[
{
"docId": "3gedef-dftrw4-gs4gfdgd-44etegre",
"filename": "/username@domain.com/filename.pdf",
"size": 1000,
"sourceFileId": "cecf5abb-9ecf-11ea-8275-5793c2ae4c49"
}
]
3. Retrieving Documents from a Stream
To fetch documents specific to a stream:
POST api/v2/streams/{streamId}/docs/list
Example response:
{
"itemCount": 1,
"pageSize": 1000,
"pageCount": 1,
"offset": 0,
"data": [
{
"docId": "3cf66840-5af6-11e9-81f2-53ce62da45da",
"sourceFileId": "205977d2-6082-11e9-81fd-5f258a7bb38a",
"state": 7,
"documentName": "filename.pdf",
"pageCount": 2,
"creationDate": "2019-04-09T18:35:25.337Z"
}
]
}
4. Creating Named Batches in a Stream
To create batches within a specific stream:
POST api/v2/streams/{streamId}/batches
5. Getting Batches from a Stream
To get a list of batches in a stream:
POST api/v2/streams/{streamId}/batches/list
Stream Permissions and Access Control
- Admin users can create and manage all streams within a project
- Standard users can only access streams they've been assigned to
- Permission settings can be configured to allow or restrict access to certain streams based on user roles
- Documents remain in their assigned stream unless explicitly moved
Streams vs. Projects: When to Use Which
| Use Streams When You Want To: |
Use Projects When You Need: |
- Maintain the same document structure and extraction models
|
- Different document structures
|
- Separate workflows while sharing project settings
|
- Entirely separate extraction models
|
- Organize work based on teams, clients, or approval stages
|
- Completely isolated workflows with no shared settings
|
Stream-specific Reporting and Analytics
- Filter dashboard views by stream to see performance metrics for specific workflows
- Generate reports for individual streams to analyze processing times and volumes
- Compare metrics across streams to identify optimization opportunities
Pro Tip: Use stream-specific analytics to compare processing efficiency between different teams or workflows. This can help identify best practices that can be applied across your organization.
Best Practices for Streams
- Use separate streams for testing and production to prevent accidental data contamination
- Assign streams to different departments or teams for better workload distribution
- Create streams for major clients or document sources to organize processing
- Use clear naming conventions for streams to ensure easy identification
- Regularly review stream usage and reorganize as workflows evolve
- Monitor stream-specific metrics to identify bottlenecks or inefficiencies
With proper use of streams, you can maintain organizational clarity while scaling your document processing workflows in SortSpoke.
