Factory source-first refresh ledger
Research started 2026-11-05 UTC against base 99eaa3d1a38ce28ed421ac4020e37e898ea98c9e.
Scope: current Factory product surfaces and common organization controls; older omissions, installed clients/companions, credentials/identity state, managed policy, approval bypass and integrations are discovery requirements. No tenants declared — Tier 0+1 only. No agent installation, credential creation or live setting changes are authorized. A credential-free official Fenic SDK was installed only in an ignored isolated test environment to verify native class fields; no Session or server was created.
This ledger was started before selecting new controls. Existing source/validation reports describe the historical 2026-10-30–31 run and are not evidence of a fetch in this run.
Baseline
One standalone Factory guide v0.1.0, 19 leveled controls, 16 pack-backed controls, 23 source packs across four types (native config, REST, CLI, official Python SDK). Identity terms identify The San Francisco AI Factory, Inc.; current ownership and acquired-product boundaries are being rechecked from primary sources.
Discovery queue
| Candidate / source lead | Plane | Evidence read 2026-11-05 | Initial disposition |
|---|---|---|---|
| Product index: https://docs.factory.com/ | All surfaces | Web fetch: App/CLI/Exec/SDK, Delegations, Automations, Workstreams, Missions, Computers, Environments, Remote Control, OpenCompute, Enterprise, API and changelog links. | Pending reconciliation; existing guide is not the product inventory. |
| Terms: https://factory.com/legal/terms-individual | Ownership | Web fetch: July 14 terms identify The San Francisco AI Factory, Inc. as provider; local software/command responsibility. | Pending corporate-change check. |
| News: https://factory.com/news | Ownership/product lifecycle | Web fetch: HumanLayer joining (Nov 4), Typedef (Oct 30), Mentlio (Oct 9). | Follow primary announcements and surviving product docs. |
| Changelog: https://docs.factory.com/changelog/release-notes | Releases/client/API | Web fetch: latest CLI/Desktop 1.4.0 Nov 5; Oct 30 organization audit API; Nov 4 child task permission-skip inheritance; Oct 27 folder trust on resume; Oct 20 Chrome cookie import. | Follow to actual configuration/contract sources; candidates, not yet final controls. |
| Remote Control: https://docs.factory.com/remote-control | Installed companion | Successful web content retrieval; detailed reading pending. | Pending identity/registration/lifecycle reconciliation. |
Retrieval limits
https://docs.factory.com/llms.txt,/llms-full.txtand/sitemap.xml: web retrieval returned inaccessible/Internal Error. Sandboxed curl failed DNS; elevated curl timed out after 20 seconds. No index content is claimed from those attempts. The rendered product index and individual references supplied successful fallbacks.
Additional source-derived candidates (before control selection)
| Candidate / precise source | Plane | Observed conditions and behavior | Initial disposition |
|---|---|---|---|
| HumanLayer acquisition: https://factory.com/news/humanlayer | Corporate / separate hosted product | Web fetch Nov 5: Factory says “As part of this acquisition”; hosted HumanLayer stays up during an unspecified transition supporting export, migration and self-hosting. | A separate control plane exists; inspect its native configuration before forming a product guide. |
| Typedef: https://factory.com/news/typedef ; https://www.typedef.ai/ | Corporate / data context | Both first-party pages acknowledge joining/acquisition; product page has only capability-level SSO/RBAC/VPC assertions, not admin steps. | Research gap for enforceable product controls; marketing is inadmissible as configuration evidence. |
| Mentlio: https://factory.com/news/mentlio ; https://www.mentlio.com/docs | Corporate / local agent and reporting | Announcement says acquired/joining and work coming into Factory; existing website points to docs. Web docs retrieval returned zero content; privacy-control link resolves to home. | Browser fallback required; do not infer inherited Factory policy or fabricate a product guide. |
| Native MCP identity: https://docs.factory.com/harness/mcp#configuration-file | Installed client / credential state | Web fetch: OAuth tokens global across projects, system keyring or fallback file; Clear Auth removes stored server credentials; env expansion fails if referenced variable is missing. | Existing 3.1 only partly covers; review explicit token lifecycle and no unsupported keyring-enforcement switch. |
| Marketplace credential storage: https://docs.factory.com/enterprise/internal-plugin-marketplaces#private-repository-access | Installed client / SCM | Web fetch: Git helper/keychain/GCM/SSH; store persists plaintext; Computer requires connected account access plus Factory App grant. |
Existing 3.2/3.4 incomplete; protect native integration credential access. |
| Marketplace installed provenance: https://docs.factory.com/enterprise/internal-plugin-marketplaces#restricting-marketplaces | Managed config / rollout | Web fetch: installed source/name mismatch checking rolls out separately; prior clients/relative-only lists have exceptions. | Existing 3.2 needs rollout limits and provenance check, not “strict list proves everything.” |
| Delegations: https://docs.factory.com/delegations#personal-and-shared-identity | Inbound apps / shared profile | Web fetch: Slack/Google Chat DMs use sender identity; shared channel/issues use profile service account; Access is Everyone, custom groups not yet available. | Missing dedicated guidance; connector enablement is a different boundary. |
| Independent review: https://docs.factory.com/delegations#require-independent-github-review | SCM integration | Web fetch: requester can approve a Factory-authored PR; two approvals or a dedicated check required to ensure another human. | Missing integration-specific boundary; source-controlled setup belongs in GitHub guide context. |
| Automations: https://docs.factory.com/automations/custom#webhook-triggers | Scheduled / Slack / webhook / GitHub | Web fetch: identity and session visibility separate; header secret preferred, query secret risks logs; raw-body filter, 10-minute identical-body dedupe, hourly run cap, pause/owner lifecycle. | Missing controls; no HMAC or arbitrary replay guarantee claimed. |
| HumanLayer daemon identity: https://docs.humanlayer.com/explanation/remote-daemons#authentication-and-persistence | Daemon / native credential state | Web fetch: refresh tokens in ~/.humanlayer/riptide/, optional home; launch credentials process-only; host ID stays at default path; daemon has OS-user access. |
Separate product candidates: protect persistence, scoped host access, lifecycle. |
| HumanLayer automation: https://docs.humanlayer.com/guide/automation-sessions | CLI / automation | Web fetch: PAT expiry setup; daemon bypasses agent approval prompts and streams to Cloud; Cloud follow-ups reset inactivity. | Separate credential and unattended-run control; no live execution. |
| HumanLayer workspace: https://docs.humanlayer.com/reference/workspace-config#copyglobs | Native config / copied secrets | Web fetch: defaults copy .env and local agent settings; lists append/dedupe and cannot remove defaults; commands execute sh -c; local overrides win. |
Missing source-first exposure control; empty copy list is not a protection. |
| HumanLayer Slack: https://docs.humanlayer.com/guide/slack-integration | Integration / outbound updates | Web fetch: admin-only connection, event/owner filters, selected channel, private-channel membership, disconnect. | Source-backed product integration control; OAuth scopes not specified by page. |
Public reference snapshots fetched through web.run for identity, runtime, SDK and telemetry are retained locally under .hth-validation/factory-refresh/. This directory is excluded from publication. Successful reference retrieval does not imply live validation.
Final product map and evidence boundary
The initial dispositions above were captured during discovery, before controls were selected. The reconciled dispositions below supersede their pending entries. Corporate statements are company-stated evidence, not independent legal closing verification. Configuration behavior is single-sourced to the named vendor reference unless another source is named; no runtime guarantee is inferred from documentation alone.
| Chain / product | Current primary evidence, retrieved 2026-11-05 | Final disposition |
|---|---|---|
| The San Francisco AI Factory, Inc. → Factory → common organization and Droid policy | Terms, documentation index, managed settings and CLI references. Web retrieval; managed-settings/CLI full browser snapshots. | Existing guide becomes Factory Common Controls, preserving all 19 original numbered titles/anchors. It governs Factory’s organization/managed Droid plane; it does not configure separate acquired consoles. |
| Factory → HumanLayer | Factory announcement: completed-acquisition language, hosted transition/export/migrate/self-host commitment, no universal EOL date. HumanLayer release notes: Nov 2 core open source and hosted service unchanged. Web plus full browser humanlayer-index.txt, hl-release.txt. |
New separate eight-control product guide with its own host, credentials, task sync and integrations. Open source availability is not a verified self-host/export/migration contract. |
| Factory → Mentlio | Factory announcement: acquisition title/team joining, routing/compression work entering Factory. Browser mentlio-docs.txt recovered actual product docs updated Sep 3, despite a zero-line web response. |
New five-control product guide from real administration/agent instructions; no inherited Factory policy. Hosted lifecycle/migration and executable administration interfaces remain unverified. |
| Factory → Typedef | Factory announcement, Typedef both describe acquisition/joining. Browser typedef.txt / typedef-links.json show product/analytics/integrations marketing and open-source Fenic link, not an admin/security configuration reference. |
Unverified product configuration gap: no Typedef guide stub or policy/API invented from SSO/RBAC/audit/VPC capability claims. The actual product remains in the inventory. |
| Typedef → Fenic library | Typedef’s first-party open-source link followed to Fenic, then its actual docs, native config/OpenRouter/MCP references and released v0.13.0 source. Browser snapshots plus GitHub file retrieval; actual released SDK classes checked locally. | New three-control library-only guide for native LLM-response cache, provider/credential destinations and MCP tool exposure. Two controls have three real config/SDK packs. No central Factory policy inheritance, generic DataFrame hardening or hosted Typedef admin claim. |
Reconciled source-derived candidates
All rows below were fetched this session. web means content-bearing reference retrieval, not HTTP status alone. browser means rendered full text in the isolated factory-refresh group; filenames refer to ignored local snapshots. Newly introduced control IDs are listed explicitly. Historical benchmark findings are not relabeled as fresh verification.
| Source / precise area | Retrieved evidence and conditions | Disposition in actual guidance |
|---|---|---|
| Managed settings: hierarchy/system policy/merge/schema | Web + browser managed-settings.txt. Org hard controls dominate; runtime/local defaults differ; system file short-circuits API, unreadable falls back, malformed/schema-invalid becomes empty policy; file paths vary by OS. |
Covered in Factory scope and 1.3,2.1–2.5,3.1–3.3,4.1–4.4,5.1; preserve fragment/rollback warnings. No local CLI preview is presented as complete schema/enforcement validation. |
| Identity and access, organization model, service accounts API | Web identity-web.txt. Domain verification, WorkOS SSO/Directory Sync, roles and separate service-account Git identity; human Manager/Owner administration. |
Covered existing Factory1.1–1.3,3.4. Enterprise Admin Console fetch failed in this run; prior exact console path retained as historical evidence, not newly observed UI. |
| Permission rules, sandbox, Shield, agent controls | Web runtime-web.txt; rules use ordered tokens and checker0 is not Allow; sandbox reads default allowed, Factory/WorkOS domains always admitted, whole-process fails without primitives; Shield staged-secret boundaries and preview features. |
Covered Factory2.1–2.5. No prefix-completeness, universal host support or preview-GA claim. |
| CLI commands and update settings | Browser cli-reference.txt: --version, update --check, exact version rollback, npm build update default with explicit env override, org disableAutoUpdate controls CLI/desktop; local daemon restart separate. |
Covered new Factory2.7 with check-only CLI read and organization config candidate; update suppression off is not proof every updater runs. No undocumented doctor invocation. |
| Release notes, Exec | Web: latest1.4.0 Nov5; Nov4 task permission-skip inheritance; unsafe Exec skips approvals, command blocks remain; Missions require High/unsafe; Oct27 resume trust, Oct20 cookie import, older fork sync/auth persistence leads. | Factory2.1 explicit inherited bypass;2.6 native identity;4.3 copies/cookies;2.7 deployed-version verification. Release-note leads followed to admitted settings/CLI sources; no guessed cookie-import org switch. |
| Native CLI security: Secure Authentication | Browser native-security.txt, found by primary-site search native-auth-search.txt. Vendor says OAuth token storage encrypted, auto-rotation30days, OS-level permissions. |
Covered new Factory2.6; vendor-stated behavior distinguished from unverified exact file, cipher/key handling, enforced keystore-only control or protection-status command. Git helpers/MCP are separate. |
| MCP: configuration/auth/persistent permissions | Browser mcp.txt: global OAuth system keyring or fallback, Clear Auth; user/project/folder files; missing supported env refs fail connection; persistent approvals fingerprint transport. |
Factory3.1 and new2.6, CLI inventory retained. No per-project token-isolation, force-keyring switch, fallback path or complete OAuth-scope claim. |
| Private marketplaces/provenance | Web: org-added sources admitted, relative-only legacy exception, installed provenance enforcement rolls out separately, installed inactive entries can reactivate, full40 SHA and mutable npm re-resolution; Git store plaintext vs keychain/GCM/SSH. | Factory3.2/3.4 expanded; strict list is not universal installed provenance proof. Git credential helpers do not establish native Droid token protection. |
| BYOK helper and legacy migration | Web: personal CLI/desktop only, local keys, env/helper support, helper user/org only, TTL5min by default, legacy config copied and retained; model routing independent. | New Factory2.6 and existing4.2; no unsupported per-project helper or web/mobile personal BYOK. |
| Delegations identity/profile/review | Web: shared Apps service account vs personal Slack/GoogleChat DM, Access Everyone/custom groups coming soon, requester may approve Factory PR. | Covered new Factory3.6 plus3.4 independent SCM review. No per-channel Factory ACL or human-identity fallback assumed. |
| Slack, Linear, Jira, Teams, Google Chat | Web integration-sources.txt: actual install/configure/consent, shared profile, channel membership, issue context and result posting; Google Chat in development. |
Factory3.6 covers actual actors/data/actions, app vs personal connector, consent growth and offboarding. Exact OAuth permission manifests, signature/replay and revocation latency remain unverified; not silently treated as secure. |
| Custom Automations: triggers/identity/webhook/lifecycle | Browser automations.txt: owner/run identity/session visibility distinct; Slack top-level/sender filters/bots/future pattern channels; header secret vs leaky query, hash-only stored, rotation invalidates; raw body filters,200KiB/60perminute/10runsperhour/10minute identical-body dedupe; feed30days lacks payload; local memory/prompt persistence. |
Covered new Factory3.7. No general replay/HMAC guarantee;200acceptance not completion. App automation API is not substituted with the distinct CI Automation API. |
| Remote Control, Computers | Web: auto desktop registration/toggle; CLI daemon remote-control/register/list/SSH/forward; outbound region relay; Ed25519 identity ~/.factory/.ssh/; privileged user can alter firewall; daemon upgrade separate. |
Factory4.4 and2.6 extended; outbound-only is not no remote access; toggle does not prove cleanup. General host firewall/image hardening excluded by scope. |
| Environment prerequisites/runtime secrets, create, manage | Web create/manage compute-sources.txt; browser environment-prerequisites.txt: paid/region/access conditions, personal+org+App Git intersections; secrets only Settings→Account→Secrets, no per-Environment fields; personal overrides org, service-account identity; no secrets in build; refresh/restart for existing processes; Environment delete leaves Computers. |
Covered new Factory4.5, plus3.4/4.4. Actual org secret-name GET added; no invented per-Environment secret ACL or build-time injection. |
| Organization API | Browser organization-api.txt: GET/PATCH computer-secrets names only; PATCH upsert/delete/createOnly, write-only values; user/control-history paging contracts distinct. |
GET read pack Factory4.5; PATCH documented available but not executed. Member/history/key packs retained. Names omit personal overrides and process access; no compliant verdict from an empty list. |
| OpenCompute, privacy/data flows | Web: self-hosted provisioning/private preview, hosted control-plane default, model inference/session storage separate. | Factory4.2/4.4/4.5 applicability/reference. Generic provider infrastructure configuration excluded; private execution is not proof of wholly private processing/storage. |
| Python SDK, TypeScript SDK | Web sdk-and-data-web.txt: official agent/session SDKs, discovery list_models retained from existing real Python SDK pack. |
Factory4.2 official SDK read retained; no new SDK implementation fabricated for org admin. SDK compatibility/live catalog remains untested; offline substitution is explicitly only a test fixture. |
| Telemetry, privacy | Web sdk-and-data-web.txt: org-only sink/literal credentials, raw content opt-in, aggregate metadata and separate Factory collector. |
Factory5.1 retained; no anonymous aggregate claim or invented SIEM table. |
| Audit log, API index | Web index resource groups/current audit page and October30 announcement: Enterprise Owner audit access. Current reference did not establish an exact general audit export route/schema; guessed audit URL attempts failed and do not prove absence. | Factory5.2 keeps real paginated settings-history read and console audit path; explicit general audit contract gap. No general audit exporter or SIEM event schema invented. |
| HumanLayer daemon explanation, setup | Web: OS-user authority, saved refresh tokens under riptide, relocated credential home vs fixed host_id, beta/prod separate, ephemeral launch credentials, supervisored restart and outbound API/sync/WorkOS hosts. | HumanLayer1.1; native logout writer guarded. Exact at-rest enforcement/status, general org policy/API and mounted-credential lifecycle remain unverified. |
| HumanLayer local daemon, release history | Web + browser hl-release.txt: desktop-only switch scope, stop-failure nuance, task persistence, queued offline hosts; pinned CLI self-update flag/env and compatible version. |
HumanLayer1.2; no universal account-wide off switch or fabricated org host allowlist. |
| HumanLayer workspace reference | Web: strict JSON/local overrides; default HEAD; setup shell after copies; setup blocking false; default env/local-agent-settings copy globs append/dedupe, cannot remove with empty list. | HumanLayer2.1 config generator and explicit clean-source guidance. No empty-copy-list pseudo-control. |
| HumanLayer automation sessions | Web: personal PAT expiry, CLI argv contract, ephemeral daemon bypasses approvals, default2minute inactivity reset by interactions, stop shutdown option, no clone/workspace preparation. | HumanLayer2.2; no launch/token creation/test live. Release notes label initial GitHub Actions access experimental; current universal GA/entitlement unverified. |
| HumanLayer gateway | Browser hl-gateways.txt: agent direct egress; daemon environment, Claude settings, Codex CODELAYER endpoint vars, HTTPS/loopback exception, proxy limits and restart. |
HumanLayer3.1 config generator (deliberately stricter HTTPS-only); downstream agent policy not duplicated or claimed inherited from Droid. |
| HumanLayer GitHub, Linear, Jira | Browser hl-github/linear/jira.txt: selected repositories and exact GitHub read/write permissions, task imports/artifact comments; Linear/Jira enable vs disconnect, auto-advance; real HumanLayer Linear companion read/write/dry-run/destructive flags. |
HumanLayer3.2, actual first-party Linear read pack. Exact Linear/Jira OAuth scopes and revocation latency remain unverified; no broad auto-archive write. |
| HumanLayer Slack | Web: admin connect, destinations/owner-event filters, private membership and disconnect. | HumanLayer3.3; outbound notifications explicitly distinct from Factory inbound delegation; OAuth scopes/API/signature/audit gaps explicit. |
| HumanLayer task model | Browser hl-tasks.txt: task-level access/edit, local↔cloud sync on supported agent file operations, text10MB/object storage binaries, downloads on another host. |
HumanLayer4.1; default audience/exact picker/admin enforcement/retention/export/deletion remain unverified, not secure-by-assertion. |
| Mentlio product docs | Browser mentlio-docs.txt: role/team review, local Cursor identity, derived telemetry, manager routing/savers, Integrations reconnection/removal, caps only via managed hooks/proxies. |
Mentlio1.1,2.1–2.2,3.1,4.1. Company privacy/enforcement statements are not tested. No complete roles/scopes/config/API/MDM/native state contract established, so zero invented packs. |
| Fenic native config, v0.13.0 actual source | Browser fenic-config.txt; source blob f908f0d4797bf7766de1fb9cffcdd5656d4901ad. Released SDK confirms LLMResponseCacheConfig has backend/ttl/max_size_mb/namespace, no enabled; enabled=False silently ignored; SemanticConfig.llm_response_cache=None disabled. Seconds/minutes TTL branches do not consistently enforce stated 30-day/minimum bounds. |
Fenic1.1 disabled config and separate read-only actual SDK inspection. Historical deletion, cache path/backend, encryption, DataFrame/catalog/provider storage remain separate unverified contracts. Cache-object defaults do not mean caching is globally enabled by default. |
| Fenic OpenRouter, actual provider classes | Browser fenic-openrouter.txt: provider env identity, routing/fallback/PDF parameters. Source/released class uses ignore, topic shows exclude; actual nested Provider forbids extras. Whole SessionConfig semantic-model validation can fetch OpenRouter model catalog, observed DNS-denied during isolated check; no model inference or Session occurred. |
Fenic2.1 covers every profile/embedding/custom endpoint/cloud boundary. SDK generator uses actual class only, explicit only, denied data collection and no fallback, without model-bearing SemanticConfig or live catalog validation. No data-retention guarantee from a routing request. |
| Fenic MCP, CLI, server helpers, SystemToolConfig | Browser fenic-mcp.txt; source blobs 61cbb9b8d645de48acfe823a5ffecad62d527011, 66adf924808b1ac48823f3830c245c7466f29316, 7497a90a44ab34e17c5e85f6a7f82a28930d4ac6. Tools/tables, Read/Search/Analyze, row limits, CLI listener and transport documented; public documentation MCP distinct. |
Fenic3.1 manual native exposure review; no live server, guessed caller ACL/auth configuration or generic gateway pack. CLI/source available is not equivalent to hosted admin or safe read-only enforcement inspection. |
Verified material findings (Phase 2 format)
| Type | Finding / guidance change | Primary evidence | Target |
|---|---|---|---|
| OWNERSHIP | Reconcile Factory’s acquired HumanLayer/Mentlio/Typedef and separate active control planes; do not equate ownership with policy inheritance. | Factory acquisition announcements and surviving product docs above | Hub and product overviews |
| OMISSION | Native client credential states differ; vendor encrypted-login statement needs its own bounded evidence, not Git-helper inference. | Native security, MCP and BYOK references | Factory2.6,3.1–3.2 |
| OMISSION | Installed marketplace provenance rollout/legacy exceptions and mutable npm refresh weaken a simplistic strict-list claim. | Internal plugin marketplaces | Factory3.2 |
| OMISSION | Delegated shared vs personal identities, Everyone access and independent SCM review need explicit treatment. | Delegations and app setup docs | Factory3.4,3.6 |
| OMISSION | Persistent automation trigger selection, webhook secret lifecycle, raw-input trust and privacy need their own control. | Custom Automations | Factory3.7 |
| OMISSION | Account-scoped runtime secrets, overrides and refresh/restart differ from Environment build scripts. | Environment prerequisites + Organization API | Factory4.5 and API read pack |
| OMISSION | Client/daemon update channels and org disableAutoUpdate were not inventoried by the earlier guide. |
CLI reference + managed settings | Factory2.7 CLI/config |
| MATERIAL DELTA | Permission skipping now propagates to child tasks; session/autonomy defaults do not restore skipped prompts. | Nov4 release notes + Exec reference | Factory2.1 |
| OMISSION | HumanLayer native credential persistence, default worktree secret copies and approval-free automation require distinct controls. | HumanLayer native references | HumanLayer1.1,2.1–2.2 |
| OMISSION | Mentlio local identities, derived telemetry, routing and direct-client bypass require their own source-bounded guidance. | Mentlio product documentation | Mentlio guide |
| OMISSION | Fenic native response cache and provider/MCP boundary are substantive product settings despite its library form. | Actual Fenic docs and released Python SDK | Fenic1.1,2.1,3.1 |
| MISCONFIGURATION | Published enabled=False cache example is ignored; provider exclude example conflicts with actual ignore field. |
Versioned source and actual released classes | Fenic1.1/2.1; reject ignored cache keys rather than issue a false clean verdict |
Six-surface census and read/write decisions
Available means the named contract is documented, not exercised live. A missing source is unverified, never verified absence. This matrix covers every implemented pack handle and groups controls sharing the same evidence boundary.
| Surface | Factory | HumanLayer | Mentlio |
|---|---|---|---|
| Terraform resource and data-source catalogs | Unverified vendor-native catalog. Official product/API/config indexes did not establish one; fresh full registry resource/data-source verification incomplete. No generic VM provider substitute. | Unverified current vendor-native resource/data-source catalog. | Unverified current resource/data-source catalog. |
| REST and GraphQL | Available REST: paginated members/control history, service accounts/key metadata and Computer secret-name GETs implemented as reads. Documented user/limit/service-account/secret writes not executed; they do not substitute for managed settings, App automation or delegation setup. Current index lists sessions, CI automations, Computers, Environments, org, service accounts and AutoWiki. General audit route/schema and GraphQL remain unverified. | Core API open sourced per release, but supported hosted admin REST categories/GraphQL schema not established. No guessed task ACL, retention or integration writer/read. | Supported admin REST categories and GraphQL schema unverified; no guessed routes. |
| First-party CLI/daemon | Available droid runtime/rules/MCP/plugin/tools/Computer/version/update-check reads; daemon registration/SSH/forward and update writers documented but not executed. Deprecated legacy command lists remain respected; native rules/strict plugin provenance require client/rollout checks. |
Available humanlayer login/logout/daemon commands, native self-update flags; automation entitlement/GA unverified. Available HumanLayer linear companion workflow-state read and state writes; read pack only. Logout writer dry-run by default. |
Local desktop agent available; public first-party command reference/daemon flags unverified. No unofficial OS command filed as CLI. |
| Official SDK | Available Python and TypeScript session/agent SDKs; real existing Python model discovery read retained. No org/security admin method established. No SDK install or live model catalog validation. | Current hosted admin SDK contract unverified; historical approval SDK branding not substituted for this product. | Current admin/security SDK contract unverified. |
| Native config/system/MDM | Available org JSON policy/schema and protected OS managed paths, local settings/MCP/BYOK, hooks/daemon flags. 15 config packs (12 JSONC+3 generators), fragments only. No configurable native login-keystore enforcement or cookie-import org switch verified. | Available strict workspace JSON/local overrides, daemon/provider env and CLI flags; two config generators. No current org MDM/keystore-only policy verified. | Manager/desktop UI controls available, config schema/MDM/native credential paths unverified; no pseudo-policy file. |
| SIEM / native queries | Available OTLP exports, audit console and settings history. General audit-event schema/export route and deployed SIEM schema unverified; no invented index/table/Sigma. No vendor-native DB query plane established. | No stable general audit exporter/event schema verified; no assumed SIEM table. | Derived telemetry/dashboard and notifications documented, audit export/schema unverified. |
Deliberate exclusions and unresolved limits
Fenic’s six-surface census is separate from the three hosted/agent products above: Terraform resource/data-source catalog and hosted REST/GraphQL administration remain unverified; official Python SDK and native SessionConfig JSON are available v0.13.0, with a config setter, separate local cache read and provider-fragment generator; official fenic/fenic-serve commands are available, but no server is launched as an audit; SIEM audit schema/export remains unverified, and ordinary catalog SQL is outside scope. No verified TypeScript SDK equivalent was found; Python is a necessary official-SDK exception, with new test orchestration in Bun/TypeScript. This does not transfer Factory managed/MDM policy to the library.
- Excluded: generic host/VM/image/database/firewall configuration, arbitrary application source hardening, model quality and cost tuning without an access/data/enforcement objective. Product-owned credentials, host identity, native policy and persistent task data stay in scope.
- Not newly verified: product-specific CIS, DISA, CISA SCuBA, NSA/ACSC baselines and Tier3/4 product incident coverage for HumanLayer/Mentlio/Fenic. No source-only absence claim, fabricated benchmark mapping or unverified incident is published. Factory’s older benchmark-search results remain historical. These source areas are explicit gaps, so this refresh does not claim exhaustive discovery.
- Failed then recovered: Mentlio
/docsweb0lines → full browser7164characters; managed settings/MCP/custom Automations full browser replaced thin/truncated retrieval; browser navigation timeouts were checked by read before continuing, with no failed attempt labeled successful. API/admin/SDK live validation remains unavailable because no tenant or credentials are declared. - Remaining contracts needed: hosted Typedef administration; Fenic physical response-store path/purge/encryption, cloud identity and remote MCP authentication/ACL contracts; exact Droid login-cache protection/status details; full provider resources/data sources/GraphQL; general Factory audit route/schema; App automation/delegation administration and consent/signature contracts; HumanLayer hosted default sharing/admin identity/retention/export/migration and feature entitlement; Mentlio scopes/roles/native state/MDM/API/lifecycle. Capability marketing and inaccessible references are not implementation evidence.
Authored-corpus reconciliation
Factory: 24 controls,18 pack-backed,26 packs across4 types (15config,4REST,6CLI,1official PythonSDK). HumanLayer: 8 controls,4 pack-backed,4 packs across2 types (2CLI,2config). Mentlio: 5 controls,0 packs with a per-control manual/unverified automation verdict. Fenic: 3 controls,2 pack-backed,3 packs across2 types (1config,2official PythonSDK). Combined: 40 controls,24 pack-backed,33 source packs. Helpers are excluded. These counts describe the authored corpus; they do not prove all relevant controls were discovered or any tenant was hardened.