Local File Watcher

What it ingests

Files from a directory you mount on the Corveil workers host. Each file becomes one CreativeWork in the ontology, carrying its relative path, size, content type and modification time.

This is the source for drop folders, NFS/EFS shares, and air-gapped export directories — the same shape as the Amazon S3 sensor, when the files live on a volume rather than in a bucket.

Body text is extracted from text-like formats — .md, .txt, .csv, .tsv, .json, .yaml, .xml, .html, .log, .sql, .vtt, .srt — plus .pdf and .docx. Everything else still lands as an entity with its path, size and metadata, just without a body.

This is not a host-wide file search

The sensor never reads an absolute path you type into the dashboard. The operator mounts one volume (FILESYSTEM_SENSOR_ROOT); each org is confined to {root}/{organization_id}/. A config path is a relative subdirectory of that org folder.

Credential

None. Reachability is the mounted volume. There is no vault type to create.

Operator setup

On the workers host (the process that polls sensors):

1. Mount the ingest volume at an absolute path, e.g. /data/corveil-ingest.
2. Set FILESYSTEM_SENSOR_ROOT=/data/corveil-ingest.
3. The directory must already exist. Corveil creates {root}/{organization_id}/ (and the configured subdirectory) at 0700 on first Test connection; it will not create the operator root itself.

Unset FILESYSTEM_SENSOR_ROOT does not hide the sensor from Add sensor. Validate and Poll fail closed with a message naming the env, so a SaaS deploy with no volume does not silently walk the container's writable layer.

On ECS Fargate / similar, this source is useful only when the task definition mounts EFS (or equivalent) at that path. For files that already live in a bucket, use Amazon S3 instead.

Config fields

Key	Required	Notes
path	no	Relative subdirectory under this org's ingest folder. Blank ingests the org folder itself. Never an absolute host path.
recursive	no	true (default) descends nested directories; false ingests only files directly in the path.

What is skipped

• Any path component starting with . (.git, .env, .ssh)
• Symlinks (they are not followed, so a link pointing outside the org folder cannot leak)
• Well-known secret extensions: .pem, .key, .p12, .pfx, .p8, .kdbx

A rewrite of a file is a new event (the event id carries a content hash) and converges on the same CreativeWork (the entity id is the stable filesystem: path).

Wire-up

POST /api/sensors

Org-root drop folder:

{
  "sensor_type": "filesystem",
  "name": "Knowledge drop",
  "config": {}
}

A subdirectory, non-recursive:

{
  "sensor_type": "filesystem",
  "name": "Weekly reports",
  "config": {
    "path": "reports/weekly",
    "recursive": false
  }
}

Drop files into {FILESYSTEM_SENSOR_ROOT}/{organization_id}/reports/weekly/ on the workers host.

Verify

Run Test connection. It resolves this org's ingest directory, creates it if needed, and confirms the configured path is a readable directory — the same confinement the poll loop uses.

Then drop a .md file into the directory and wait for the next poll (or trigger Poll now). A CreativeWork named after the file should appear in the graph, with the file body as source text.

Poll, not inotify

Each tick walks the tree and watermarks on mtime, the same shape as S3. A file that lands while Corveil is down is picked up on the next sweep. Latency is the poll interval (default 5m). A true filesystem watcher can be added later as a push front end that shares this mapper.