Import Collections - Parser Architecture¶
Developer reference for Vayu's import subsystem: how raw Postman / Insomnia / OpenAPI files are detected, parsed into Vayu's internal draft model, and persisted.
The parse is the engine's since issue
#877. Every format below is read by
engine/src/core/import_document.cpp, behind
POST /import/parse, and the
app calls it. There is no parser in app/src any more, and that is the point:
the renderer's stack was the second reader of a document, which is what
#853 set out to end - and while it
existed, an agent over MCP could bind, diff, sync and export a spec but could
not import one. It can now (import_document, see
mcp.md).
Code: engine/src/core/import_document.cpp (the parse),
app/src/services/importers/ (the batch ledger, the temp ids and the apply).
This folder documents each format's mapping rules - what a Postman
authblock becomes, which OpenAPI response becomes an example. Document behavior from the code:engine/src/core/import_document.cppand, for the two OpenAPI dialects,engine/src/core/openapi_drafts.cpp(the request builder those rules are shared with) win when in doubt.
Per-format docs¶
| Format | Detection (summary) | Doc |
|---|---|---|
| Postman Collection v2.1 / v2.0 | info.schema contains v2.1.0 / v2.0.0 (or info+item with no schema) |
postman.md |
| Postman Environment / Globals | _postman_variable_scope is "environment" or "globals" && values is an array |
postman-environment.md |
| Insomnia Export v4 | _type === "export" && __export_format === 4 |
insomnia-v4.md |
| Insomnia Export v5 | type is collection.insomnia.rest/5.x or environment.insomnia.rest/5.x (YAML); a design document, a mock server or another format version is refused by name |
insomnia-v5.md |
| OpenAPI 3.0 | openapi starts with 3. |
openapi-v3.md |
| OpenAPI 2.0 (Swagger) | swagger is 2.0 (string or number) |
openapi-v2.md |
JMeter .jmx test plan |
the raw text contains <jmeterTestPlan |
jmeter.md |
Pipeline overview¶
A raw import string flows through three stages, once per document:
picked files ──▶ detectBatch() ──▶ parseImport() ──▶ assignTempIds() ──▶ ImportOrchestrator.run()
(batch.ts) (factory.ts) (assign-ids.ts) (orchestrator.ts)
│ │ │ │
one row per file, POST /import/parse stamp opaque temp flatten to one
bundle its $refs → ImportResult ids (c1/r1/e1) POST /import/apply
Each stage that waits reports how far through it is
(issue #882). detectBatch and
reparseBatch take an onProgress callback and tick as each document's promise
lands - the waves stay parallel, so what advances is a counter, never a queue -
and ImportModal adds the two stages either side of them: the URL download
(bytes, streamed off POST /import/fetch) and the apply, which is one
transaction per file and so counts files. Reading the picked files off disk gets
a name but no counter: FileReader is local and effectively instant, and a
number there would be motion without information. Every total is a real count of
real work - parsing's is the documents that will actually be parsed, which is
fewer than the documents picked whenever one could not be read or was inlined
into another.
An agent has no dialog, so it takes the whole of that in one call:
POST /import is the parse, the
flattening and the apply together. The app deliberately does not use it - a
person picks which files of a batch to import and toggles two options between
the parse and the apply.
0. The batch - batch.ts¶
Entry is per file, not per import: the File tab takes a multi-file drop or
selection, and "Import folder" takes a whole directory (webkitdirectory,
recursing, filtered to IMPORTABLE_EXTENSIONS - .json / .yaml / .yml /
.jmx). The
URL and Paste tabs are single by construction and travel the same path as a
one-entry batch, so there is no second flow beside this one to drift from it.
A file dropped anywhere on the window opens the dialog with it (#2252),
the way a Dock or taskbar drop does (#1364): useWindowFileDrop queues the
dropped Files on import-modal-store (openWithFiles) and ImportModal
takes them once into the File tab's handleFiles, so a window drop is another
source of the same batch, not another flow. The listener is bubble-phase on
window and skips an event something already cancelled, because the dialog's
own dropzone and the binary body panel handle their drops themselves. A drag
that is not carrying files (selected text, a tab) is left alone.
The Paste tab takes one other thing that is not a document: a curl or wget
command (#2250). It is a request, not a collection, so handlePaste runs
importCommand first and a hit never reaches detectBatch or the engine, whose
answer would be "Unrecognized format". The dialog offers Open as new request
instead, which hands presetFromCommand (curl-request-preset.ts, the stored
shapes the builder's save sends) to useNewRequest, closes, and toasts
droppedFlagsNotice for any flag the command carried that Vayu cannot, as the
URL bar's paste does.
detectBatch(documents, opts, intake) returns one BatchEntry per picked file,
in pick order, and every picked file gets a row - that is the property the
whole module exists for. The dialog used to read e.dataTransfer.files[0] with
no multiple on the input, so dropping a folder's worth of specs imported the
first and discarded the rest without a word (issue #666). A row saying
"Unrecognised format" is the flow working; no row is the bug.
An entry carries the bundled raw text, its result or its error, the
unresolved-ref count, and its included state. Three row states are not
failures and are not errors:
| Row state | What it means |
|---|---|
error |
Unreadable, unparseable, unrecognised, or over the spec byte cap. Listed, unchecked. |
bundledInto |
Another picked document inlined this file as a $ref target. It is part of that spec, so it is never applied on its own - which is also what stops a split spec importing twice. |
| parsed but empty | Parsed fine, but the options left nothing to create. Included stays available; Import stays disabled while it is the only file. |
Siblings come from the batch first. bundleExternalRefs reads a referenced
file through the gated specFile:read IPC (which reads beside a document the
user picked, dropped or opened, #2357; a spec bound before that rule needs one
more pick before it can be re-read); when that file is already in the
batch its text is right here, so the batch answers and nothing touches disk. Ref
targets are resolved with the bundler's own joinRelative / dirOf against the
entry's webkitRelativePath, rather than a second normalizer that could disagree
about ...
Apply is per file, sequentially - each entry is its own
POST /import/apply transaction, so a seventh file the engine refuses cannot
roll back six good ones, and each lands as its own root collection (exactly what
N manual imports produce). The ledger states each outcome; an applied row is
unchecked and disabled, because the route is create-only with no idempotency key
and a second send is a second copy of the tree. A combined single-payload apply
was considered and rejected: all-or-nothing across unrelated files is the wrong
failure mode, and per-file leaves the engine contract untouched.
A spec that is already bound forks before the apply (issue #680). When
Import is pressed, every entry carrying a spec document is looked up against the
collections that bind one - by spec_documents.source_url, and by the stored
bytes - and a match offers Sync in that collection instead of importing a second
copy. The lookup lives in services/openapi/bound-spec-match.ts, reads the bound
documents through useBoundSpecReader (the Spec tab's own query cache), and runs
at Import rather than at detect time: it is a round trip per bound document, and
a preview should not pay for one. Import anyway does exactly what Import did
before it existed - the dialog is a fork, never a block - and an import with no
spec in it never reaches the lookup at all. See
OpenAPI Collections.
Zip import is a named non-goal. No major tool exports collection zips today (Postman exports single JSON files; vendors ship git repos). Reopen when a real source ships them.
1. Detect + parse - factory.ts → POST /import/parse¶
parseImport(raw, opts, source?) is one call to the engine. It is async since
877; everything that consumed it - the batch, the modal's option toggles -¶
awaits.
Engine-side, in core::parse_import:
- One read -
core::read_document: JSON first, YAML second. The same reader behindPOST /specsandPOST /specs/describe, which is what makes "exactly one parser has an opinion about a document" true. - Detection in a fixed most-specific-first order:
Postman v2.1 → Postman v2.0 → Postman environment/globals → Insomnia v4 → Insomnia v5 (or its named refusal) → OpenAPI 3.x → OpenAPI 2.0- the order the renderer'sPARSERSlist ran in, so the same bytes are claimed by the same format they always were. - No match →
400 Unrecognised format, whichfactory.tsturns back intoUnrecognisedFormatError. Kept apart from a document that claimed a format and is broken (400 Couldn't read the document: …): the dialog says different things about the two. - Joins each request's enabled params into its
url- see The url/params invariant below.
The two OpenAPI dialects are not a second reader either: their requests come from
core::import_drafts_of, the same builder the OpenAPI sync diff compares a
stored request against (issue #865), with the import composing the root
collection, the folders and the skip tally around it. A draft the two built
differently would report "the document changed this request" about a document
nobody edited.
The move is pinned, not promised. engine/tests/fixtures/import-conformance.json
holds, per format, what the renderer's own parsers produced for a corpus of
documents - recorded at the last commit that had them - and
engine/tests/import_parse_test.cpp asserts the engine produces the same thing
on every build.
The url/params invariant¶
A stored request carries its enabled query inside url; params[] mirrors
it for the editor, disabled rows included (see
request-storage-design.md). url is what
every execution path sends, and no send path reads a query row of params[].
An import item states params, so it is stored as the parser wrote it; only a
write that leaves params out has its rows derived from url (issue #3551).
A path row ("in": "path", issue #1764) is the exception and never joins the
query: composition writes its value into the URL's :name segment instead.
Every parser states the query some other way - Postman splits it out of the URL
into params[], Insomnia keeps a parameters[] beside a verbatim URL, the
OpenAPI parsers synthesize params for a URL that never had a query - so an
imported request used to go on the wire with its query missing, silently, until
the user happened to edit the Params table once (issue #590).
parse_import closes that with one pass over every request draft,
join_params_into_urls (engine/src/core/import_document.cpp). Each format
joins with its own rule (issue #1771): a Postman collection with Postman's query
rule, the one the Params table writes (core::encode_query_component, see
URL handling); Insomnia and JMeter with
encodeURIComponent, because neither uses Postman's set and a raw + would
read as a space; a request with disableUrlEncoding as written. The OpenAPI
parsers join their own drafts, also with encodeURIComponent, because the
spec sync diff compares stored URLs against that form. It appends, so a URL
that arrived with a query of its own keeps it -
which is what Insomnia's own send does with its two sources. The query is
written before a #fragment the URL carries, never inside it (issue #2541). Each parser's
mapping stays as documented: params[] is still exactly what the source
declared, and only the enabled rows reach url.
Two consequences worth expecting:
- A row disabled in the source stays in the table and out of the URL.
- An Insomnia, JMeter or OpenAPI row with a key and an empty value joins as a
bare key (
?verbose), as Insomnia's own join writes it; a Postman row joins asverbose=, and only a"value": nullrow asverbose(see URL handling). The OpenAPI parsers import an optional value-less parameter disabled so this does not happen for a parameter the spec merely documents - only arequiredone, or one carrying a declared value, reaches the URL. Declared header parameters follow the same rule (see Parameter values & enabled state).
2. Assign temp IDs - assign-ids.ts¶
assignTempIds(result) walks the draft tree and stamps every collection, request, and
environment with a temp id (c1 / r1 / e1 counters) in place, before the
import is sent. These are opaque per-call strings, not record ids: they exist so one item
in the payload can reference another (parentTempId, collectionTempId) while no real ids
exist yet, and they are never stored. The orchestrator throws if it sees an unstamped
draft.
This used to mint real col_… / req_… / env_… UUIDs, because the orchestrator created
items one POST at a time and had to wire the tree itself - which is the only reason
POST /<resource> ever accepted a client-supplied id. It no longer does: since #97 a
create carrying an id is a 400, on the single-resource routes and per item in
/import/apply alike.
3. Persist - orchestrator.ts¶
ImportOrchestrator takes an injected ImportApi (easy to fake in tests) and exposes
run(result, opts), which flattens the draft tree into one
POST /import/apply call:
- Collections are flattened depth-first - collection → its requests → child
collections - carrying
parentTempId(nullfor a root) and, below a root, anorder: a parent's child collections and requests share one sequence (issue #2535), placed by each draft's ownorderwhere the parse stated one (a Postman item's or a JMeter element's document position, an Insomnia item's rank by sort key, a Vayu-written document's stored order) and otherwise folders first, then requests (draft-siblings.ts, the engine'ssibling_positionsthe other copy). Each request carriescollectionTempIdandbodyType = body.mode(the engine never derives this). - Environments are included only if
opts.importEnvironmentsis true. - The engine generates every real id and returns an
idMapkeyed by temp id. The orchestrator checks that map covers every item it sent and throwsImport incomplete: …otherwise - a silently skipped item would otherwise read as a clean import. Note this throw happens after the tree is committed. The real ids go two places:useImportMutationinvalidates the collection / request / environment / globals queries, which refetch, and hands the root and environment ids back to the dialog (see After the apply below). - No rollback, and no retry. The engine write is atomic (validation over the whole
payload, then one transaction), so a rejected payload persisted nothing and the error
the modal shows is the engine's, naming the item that broke. The engine names it by
tempId(inside the error object - see api-reference);importFailureMessage(services/importers/failure-message.ts) resolves that back to the name shown in the preview, sincec37means nothing to whoever chose the file. The old per-item loop needed best-effort deletion of already-created roots; that code is gone. What atomicity does not cover is a lost response:/import/applyhas no idempotency key and mints fresh ids per temp id on every call, so a second attempt after a committed-but-unanswered write is a second copy of the whole tree.useImportMutationtherefore setsretry: false, overriding the QueryClient'sretry: 1mutation default, and invalidates inonSettledrather thanonSuccessso a tree that landed behind a later failure is still visible. - Globals are the one write outside that payload. They are an engine singleton behind
POST /globals, not a tree item with a temp id, soapplyGlobalsruns as a second request after the apply and its id-map check - nothing may fail behind it, since it is the only write here that can destroy data the import did not create. The trade is the one partial outcome left: the tree lands, the globals write fails, and the error surfaces with no rollback to undo the apply. See postman-environment.md. clientCertificatesis best-effort, inside the same call but outside the tree's own atomicity (issue #1656). A Postman import resolves a request's owncertificateinto aclient_certificatesregistry candidate at preview time (see postman.md) and the preview lists it. The orchestrator sends it alongside the tree,[]included likespecs, but only the candidates the user ticked (ImportApplyOptions.selectedClientCertificates, indexes intoImportResult.clientCertificates; issue #2363): a registry row is host-wide and outlives the import, and the document chose its host and path. A caller that states no selection sends none, and the candidates left unticked are tallied in thecertificatenotice. The engine applies each candidate it is sent through the same check-and-writePOST /client-certificatesuses, but only after the tree's own transaction commits - reusing that write means reusing its own lock, and taking it from inside the tree's transaction would deadlock. A candidate that fails (a(host, port)another row claims by the time apply runs, a file that stopped being readable since the preview) is skipped rather than undoing a tree that does not reference it; nothing in the response says which.proxyhas no equivalent - Vayu's proxy config is workspace/run-scoped, so it stays a permanentmeta.skippedtally instead.
4. After the apply - import-report.ts¶
The dialog closes on a full success and announceImport takes over (issue
#2109). The preview's per-file
ledger is the one place the loss report is drawn from, and the dialog discards
it, so what the user would otherwise have to finish by hand is kept:
useImportMutationresolves with{ idMap, rootIds, environmentIds }: each root collection's (and each created environment's) temp id mapped to the real id the engine minted. That is what the steps below address.- One
ImportReportper applied file (the notice lines, the file-reference needs, the counts) goes intoimport-report-store, keyed by each root collection's real id. A file with nothing to report stores nothing. - Every new root is expanded in the sidebar and the first one opens as the active tab.
- One toast says what arrived:
Imported "Pet Store" - 60 requests, 4 nested collections, orImported 3 filesfor a batch, with; N things to finishappended when a report has action-tier lines or file references to relink. When there is a report, the toast's one action opens the collection on its Info tab, even when the user is already inside it on another sub-tab. - If the whole import created exactly one environment and the engine reports none active, it is activated, and the toast says so or offers the next step in place of the action slot (the notes action keeps the slot when there is one). See postman-environment.md.
- The Info tab of each new root carries an Import notes callout
(
CollectionDetail/ImportNotesCallout.tsx): the same notice lines and file-reference list the preview drew, under the file's name. It stays, past the toast and the session's other tabs, until its dismiss button is pressed; that drops the report for every root the file created. It is in memory only, so a restart clears it.
A failed apply is unchanged: the dialog stays open with a per-row outcome and none of the above happens, since part of the batch may not have landed.
The flattening exists twice, and the two are pinned to each other.
POST /import has no preview to show, so it flattens engine-side
(core::import_apply_payload); this orchestrator flattens a result a person has
previewed and filtered. Two implementations of one mapping is the defect this
repo keeps finding, in the one place a missing field is silent - a request that
imported without its examples looks exactly like a document that documented none
- so orchestrator.payload-conformance.test.ts and the engine's
import_parse_test.cpp build the payload for the same fixtures and must agree.
Reading a document without parsing it¶
One renderer path still needs a document as a tree: ref-bundler.ts, which
inlines the files a multi-file OpenAPI document references before anything is
parsed or stored (issue #649). It reads through
POST /import/document -
core::read_document and nothing else - so it holds no parser of its own, and
js-yaml is gone from production src/.
That is deliberate, and it is the line the bundler sits on: fetch-time $ref
assembly is deterministic re-serialization, not an opinion about what a document
declares. It stays renderer-side because it reaches the network through
POST /import/fetch and the local filesystem through the main-process-gated
specFile:read IPC (granted documents only, #2357), and an engine that read sibling files off disk on an
import's behalf would be a wider capability than any of this needs.
Draft model (the parser output contract)¶
Parsers emit drafts, not engine rows. A draft's tempId is absent until
assignTempIds runs; no draft ever carries a real record id.
ImportResult
| Field | Type | Notes |
|---|---|---|
| collections | CollectionDraft[] | Root collections (parentId = null) |
| environments | EnvironmentDraft[] | Persisted only if importEnvironments |
| globals | Record<string, VariableValue> | Variables for the globals scope, keyed by name. Not a draft list - globals are an engine singleton, so there is no name and no temp id to assign. Required, so every parser states its answer: {} for all but the Postman globals export |
| meta | ImportMeta | Counts + lossy-import signals for the Preview UI |
CollectionDraft - name, description, variables: Record<string, VariableValue>,
auth (a concrete RequestAuth - never inherit; collections are always concrete auth
sources), preRequestScript, postRequestScript, children: CollectionDraft[],
requests: RequestDraft[].
RequestDraft - name, description, method: HttpMethod, url,
params: KeyValueEntry[], headers: KeyValueEntry[], body: RequestBody,
auth: RequestAuth (inherit allowed, resolved against the collection chain at
execution time), preRequestScript, postRequestScript, followRedirects?: boolean,
maxRedirects?: number, timeout?: number, examples?: ExampleDraft[].
Both redirect fields are optional because absent means "leave the engine's default"
(followRedirects: true, maxRedirects: 10): a parser sets one only when the source
file states it, and the orchestrator forwards it to POST /import/apply only when set.
Sending an omitted false as true would follow a 3xx the user disabled; sending an
absent value as false would stop one they never touched. Producers: Postman's
item-level protocolProfileBehavior, resolved through its folders (both fields, see postman.md) and
Insomnia's settingFollowRedirects (followRedirects only - its redirect limit is an
app-wide setting, not a per-request field, see insomnia-v4.md).
timeout (milliseconds, issue #2937) follows the same rule: absent means the request
uses the defaultTimeout setting, and the orchestrator forwards it only when set. Its
producers are a Vayu OpenAPI export (x-vayu-request.settings.timeout) and JMeter (a
sampler's connect and response timeouts, or those of the HTTP Request Defaults in its
scope, summed, see jmeter.md); Postman and Insomnia state no
per-request timeout the importer reads.
ExampleDraft - name, status: number, headers: KeyValueEntry[], body: string,
contentType: string. A saved example response the source file carried next to the request
(issue #481): Postman's item.response[], an OpenAPI operation's responses. Every parser
dropped these before the engine had a table for them - not even counted as skipped. They
carry no id: the engine assigns one, and the orchestrator nests them on the request item of
POST /import/apply rather than sending a fourth top-level section, since nothing references
an example. examples is optional for the same reason the redirect fields are: a parser with
no concept of saved responses must not look like one that found none, and array order is the
stored order - a mock server answers with the first example of a matched request.
EnvironmentDraft - name, description, variables: Record<string, VariableValue>.
ImportMeta - format, fileName?, requestCount, folderCount,
folderStrategy?, environmentCount, globalCount, exampleCount,
skipped: SkippedItem[], nonExecutableAuth: number, nonExecutableAuthRequests?: string[],
nonExecutableAuthFolders?: string[], nonExecutableAuthCollection?: string,
unattachedFileParts: number, refReport? (set app-side by the $ref bundling, never by the
engine), unsupportedScriptApis?: {request, names}[].
unsupportedScriptApis is set by the Postman parser only, and only when a script uses a
Postman API Vayu does not run: the request, folder or collection and the names it uses (see
postman.md). Absent otherwise, and not a skipped kind.
folderStrategy ("tags" | "paths" | "mixed") says which rule an OpenAPI import grouped its
folders by, and the Preview states it whenever paths were involved - a folder tree the
document never spelled out must not read as one it did (issue #710, see
openapi-v3.md). Only the two OpenAPI parsers set it: a
format that carries its own folders makes no such choice, and a parser that says nothing must
not look like one that said "tags".
exampleCount totals the saved example responses across the whole tree, from
count_examples(collections) in import_document.cpp - read off the finished drafts for the same
reason unattachedFileParts is, so the number the preview promises and the drafts about to
be written cannot disagree.
unattachedFileParts counts form-data file rows the import produced with no file attached -
an OpenAPI spec documents that a field is an upload, never which file, so those rows
import complete-but-empty and the user picks the file. Every parser gets it from
unattached_file_parts(collections) in import_document.cpp, which reads the finished drafts rather
than tallying while building them, so the number and the rows cannot disagree.
Imported file paths, and the folders they sit in. Every path an import carries - a
binary body's file.src, a form-data file part's src - names a file on the exporting
machine, so the engine sends it only when it lies under a folder allowed in Settings > Files or
is a file picked here. POST /import/parse answers each with the unresolved the engine derives
from those (#2356), true for a path this machine would refuse; no stored body keeps the key. The Preview reads the drafts (fileReferenceNeeds in
app/src/modules/collections/import-notices.ts) and lists each distinct parent folder once,
with how many files it holds and an Allow folder… action (ImportFileFolders.tsx, through
hooks/useAllowFolder.ts) that opens the system folder picker, inside Electron only (it opens where the last
folder dialog ended, not in the file's folder, #2378): main allows the folder the user
confirms, since the renderer cannot name one. A collection
exported on this same machine is one choice per folder rather than a relink per request; a
folder already allowed reads Allowed. A batch lists the folders of every file it will import,
once. A folder that does not exist here cannot be chosen, so those files are relinked in their
requests instead. A path whose folder is still a {{variable}} names no folder and is left out. A
binary body that arrived with no path at all is counted beside unattachedFileParts as
"N file bodies need files", the whole-body counterpart of that line.
What came along. Under the counts the single-file Preview adds one line saying what the
import carried beyond names and URLs - Carried over: 12 variables · auth on 40 requests ·
scripts on 8 requests - so a user does not have to open requests to learn that variables, auth
and scripts survived. It is read off the drafts like the file references above
(describeCarriedOver, import-notices.ts): the keys of every collection's variables across
the whole tree; the requests whose own auth mode is anything but none, noauth or inherit
(a request that inherits is not counted); and the requests with at least one enabled,
non-blank script.pre or script.post element. A segment that is zero is left out, and a file
with none of the three shows no line at all. Environments and globals are not counted here;
they already have their own segments in the count line.
SkippedItem - { kind: "websocket" | "grpc" | "api_spec" | "unit_test" | "file_body" |
"malformed_item" | "unsupported_method" | "malformed_spec" | "example_no_status" |
"example_no_media_type" | "default_response" | "external_ref" | "duplicate_operation_id" | "cookie_param" |
"unmapped_body" | "unresolved_base_url" | "no_server_url" | "unsupported_auth" | "oauth2_dropped_field" |
"oauth2_state_regenerated" | "oauth2_unmapped_param" |
"url_without_raw" | "variable_metadata", count, requests?, names? }.
Surfaces work Vayu can't represent so the Preview can warn instead of silently dropping.
requests names the requests a count applies to, when the walk had one in hand (every
per-operation OpenAPI count, Postman's per-request counters - script_unsupported also names
the folder or collection that declares the script - Insomnia's file bodies, unsupported
auth types, cookie API keys (apikey_cookie_header), insomnia.* scripts
(script_unsupported_insomnia) and the resources it cannot import), so the Preview can say
which request to finish rather than only how many. names names what was dropped, never its
value: the keys of the OAuth 2.0 request parameters a Postman parse did not carry
(oauth2_unmapped_param, see postman.md), and
the folders or collection that declared a Postman setting Vayu cannot apply
(protocol_behavior_folder, see
postman.md). A Postman GET or HEAD that carries a body Postman
would prune, and that does not opt out with disableBodyPruning, is counted as body_on_get
and named under requests (also postman.md). How the Preview words and ranks each kind - amber for what the user must finish, muted
for what the import decided, not shown for what changes nothing sent - is
app/src/modules/collections/import-notices.ts.
Three of the kinds are not about representability: unsupported_method is an operation whose
HTTP method has no HttpMethod (OpenAPI 3's trace), and malformed_item / malformed_spec
are shapes the source file got wrong - a Postman item[] entry that is not an object (see
postman.md), an OpenAPI path item or parameters list that is not what the
spec allows - stepped over so the rest of the file still imports. The two OpenAPI kinds are
counted via ImportTally, shared by both OpenAPI dialects: they are
structural clones, and a second copy would drift. example_no_status is a fourth
non-representability case: an OpenAPI response keyed 2XX (or with a junk key) documents a
real response, but an example is served under one status line and there is no honest value
to pick, so it is counted rather than guessed at. example_no_media_type is the sibling for a
response that documents a body and yields none, a 3.x content map or a 2.0 examples map or
schema: no JSON type that yields one and no declared string or text-family example (a
schema-only text/csv, say), so the example imports empty and the Preview says so as a note,
"imported without a body" (issue #3424). default_response is the same skip for the
default key, on a counter of its own (issue #710) because it is conformant and declared on
nearly every operation of a vendor spec - the Preview does not show it at all, which is what
keeps a 568-count line from burying the one warning that needs acting on. external_ref is the fifth, and the only
kind no parser produces: a $ref naming another file that the bundling pass could not
read (ref-bundler.ts, issue #649) is counted before parse and stamped into
meta.skipped by parseImport, one per reference - because each one is an operation that
imported without the schema it declared. duplicate_operation_id is the sixth, and the only
one where nothing is dropped from the request itself: an operationId a document declares
twice is kept on the first operation and left off the second, whose recorded identity then
rests on its method and templated path alone (issue #715). Counted per repeated declaration,
because a sync follows the identity a request records, and which of the two kept the id is
not something the user should have to work out from a diff.
The last three are OpenAPI 3 kinds added by issue #719, each closing a drop that had no
counter at all. cookie_param is a parameter declared in: "cookie": Vayu's cookies come
from the jar, and folding one declaration at a time into a single joined Cookie header
would mean inventing a merge the document never wrote, so mapping them is a recorded
non-goal and naming the loss is the honest half. unmapped_body is a requestBody
declaring only media types with no Vayu mode and no binary schema - an image/png whose
schema is an object, say - which used to return { mode: "none" } on the same path as
no body at all, reporting nothing skipped; an operation that declared no body is still not
counted, because it lost nothing. A whole-file body (application/octet-stream, or a
format: binary schema) is no longer one of them: it imports as a binary body with no
file chosen.
file_body is not a loss either. Every whole-file body - Postman's file mode, an Insomnia
binary body, an OpenAPI whole-file requestBody - imports as a binary body naming its file
(the path the export carried, marked unresolved, or none), and the count with its
requests names the bodies whose file the user has to relink or choose.
unresolved_base_url is a servers[0].url that could not be made into an address a request
could reach - a {variable} the document declares no default for, or a relative URL in a
document that arrived with no URL to resolve it against (see
OpenAPI 3.0). no_server_url is the document that names no server
URL at all (no servers, an empty one, a servers[0] with no url, or a Swagger 2.0 document
with no host): no baseUrl variable is created and the preview says to add one. OpenAPI's
implicit default server / counts as no server URL too.
The last three are Postman kinds added by issue #1443, closing gaps in the same "dropped is
counted" promise. unsupported_auth is an auth.type Postman defines and Vayu has no mode
for (a non-string type, or one the schema does not list), unlike awsv4, digest, ntlm,
hawk, oauth1, edgegrid and jwt, which import as data and are reported under
nonExecutableAuth instead. An Insomnia authentication.type with no Vayu mode (asap,
netrc) is the same kind, and imports as no auth rather than inherit. url_without_raw is
not a loss: it counts a URL assembled from host[]/path[] because it carried no raw - a
mapping that changes nothing the user sends, so like default_response the Preview does not
show it; it stays counted in meta.skipped. (A fourth, path_variables, counted a :key
segment rewritten to {{key}} plus a collection variable; issue #1764 retired it along with
that rewrite - a path variable now imports as the request's own path row, which loses nothing.)
variable_metadata is a collection, folder, environment or globals variable whose description
or a meaningfully declared type (anything but secret or Postman's own default marker) was
read and discarded, because Vayu's variable record has a field for the value and the secret flag
only (see postman.md).
Supporting value types:
- KeyValueEntry: { key, value, enabled, description? } - duplicates and enabled:false
rows are preserved. A form-data part also carries contentType?, its own Content-Type.
- VariableValue: { value: string, enabled: boolean, secret? } - all values are strings.
- RequestBody: {mode:"none"} | {mode:"json"|"text"|"graphql"|"jsonrpc"|"xml", content} |
{mode:"form-data"|"x-www-form-urlencoded", fields: KeyValueEntry[]}.
- RequestAuth: {mode:"none"} | {mode:"noauth"} | {mode:"inherit"} | {mode:"bearer", token} |
{mode:"basic", username, password} | {mode:"apikey", key, value, in} |
{mode:"oauth2", config: OAuth2Config} (executable) |
{mode:"digest"|"aws"|"ntlm", config} (stored, not executed).
On a collection, none means "nothing set here" (a descendant's inherit keeps
climbing) and noauth means "send nothing" (the walk stops there) - see
Postman auth mapping.
ImportOptions semantics¶
Options are applied at parse time - parsers emit empty scripts / skip environments when
told to. The orchestrator only re-checks importEnvironments (to decide whether to persist
the environment drafts). Honoring is per-parser:
| Parser | importScripts |
importEnvironments |
|---|---|---|
| Postman v2.1 / v2.0 | Honored - scripts emitted as "" when false |
Moot - collection files embed no environments |
| Postman Environment / Globals | Moot - this shape carries no scripts | Honored - gates both environments and globals at parse time, so the Preview counts report 0 |
| Insomnia v4 | Honored | Honored - environment resources become EnvironmentDrafts |
| Insomnia v5 | Honored | Honored - the environments object becomes EnvironmentDrafts; an environment document carries nothing else |
| OpenAPI 3.0 | Ignored - never generates scripts | Ignored - never generates environments |
| OpenAPI 2.0 (Swagger) | Ignored | Ignored |
The two OpenAPI parsers take _opts and never read it (they produce spec-derived stubs with
no scripts and no environments).
Shared helpers¶
Reusable mapping rules, shared between the formats that need them. All of them
live in engine/src/core/import_document.cpp unless noted; the two the OpenAPI
dialects use - normalize_path_templates and the sampler - are
engine/src/core/path_template.cpp and engine/src/core/openapi_drafts.cpp.
The names below are those functions'.
as_string¶
as_string(v) - coerces any scalar to its string form; objects are JSON.stringify-d;
null/undefined → "". Vayu stores all values as strings.
to_var_record¶
to_var_record(vars) - Postman variable arrays ({key, value?, enabled?, disabled?})
→ Record<string, VariableValue>. disabled takes precedence over enabled; default
enabled: true. Values pass through normalize_template_vars. Rows without a key are skipped.
Each row is stamped createdAt in array order (#3252), since the stored object keeps name order.
map_key_values¶
map_key_values(rows) - Postman header/query/urlencoded arrays → KeyValueEntry[]. Sets
enabled = r.disabled !== true, normalizes each value, carries description when present,
and preserves duplicates and disabled rows.
map_postman_auth¶
map_postman_auth(auth) - a Postman auth object (collection / folder / request) → RequestAuth.
Reads the per-type detail via auth_detail, which handles both v2.1's array shape
([{key, value}]) and v2.0's object shape. Maps bearer/basic/apikey to concrete auth,
maps oauth2 to an executable {mode:"oauth2", config} via map_postman_oauth2 (below),
stores digest/aws/ntlm/hawk/oauth1/edgegrid/jwt as {mode, config} (not executed; the CONFIG_AUTH_TYPES table in postman_format.hpp), maps the real AWS wire type
awsv4 → the internal {mode:"aws", config}, noauth → none, and missing/inherit →
inherit. A collection/folder noauth is handled by collection_auth instead,
which maps it to the terminal {mode:"noauth"}.
OAuth 2.0 mapping¶
Turns each source format's OAuth 2.0 block into Vayu's typed OAuth2Config, so imported
OAuth 2.0 auth is executable (not a passive {mode, config} bag):
- map_postman_oauth2(detail) - Postman v2.1 oauth2 params, incl. grant normalization
(authorization_code_with_pkce → auth-code + PKCE; implicit → auth-code + PKCE; a minimal
export with only a pre-fetched accessToken → a bearer token, or an API key when Postman
would place or prefix it otherwise).
- map_insomnia_oauth2(auth) - Insomnia's camelCase oauth2 object.
- map_openapi_v3_oauth2(scheme) / map_swagger_oauth2(scheme) - pick the first usable flow from an
OpenAPI v3 / Swagger v2 oauth2 security scheme (client id/secret seeded as {{variables}}).
Grant/field normalization is shared here so the parsers agree. Only the data-only modes (digest/aws/ntlm/hawk/oauth1/edgegrid/jwt)
remain non-executable and are reported in meta.nonExecutableAuth: Postman and Insomnia count the requests whose resolved auth is one of
them (declaring folders and the collection go in nonExecutableAuthFolders / nonExecutableAuthCollection; see
Postman and Insomnia v4, which also says why a folder set to No Authentication ends the walk).
postman_raw_body¶
postman_raw_body(content, language) - Postman raw body → RequestBody. json/text/xml map
directly; any other declared language is text whatever the content looks like (as Postman
sends it), kept as rawLanguage, and html/javascript also gain their Content-Type row
(with_required_content_type). Only a body with no language sniffs via JSON.parse (success →
json, else text), never guessing xml, with rawLanguage: "". Public in import_document.hpp: the Postman exporter calls it to decide
whether a kept rawLanguage still describes the stored body.
join_exec¶
join_exec(event) - a Postman event entry → a single script string. Joins
event.script.exec[] with \n (or returns a string exec as-is).
normalize_template_vars / normalize_path_templates¶
normalize_template_vars(text) - normalizes foreign template syntax to Vayu {{var}}:
{{ x }} / {{ _.x }} → {{x}} (trimmed, _. prefix stripped). Its sibling
normalize_path_templates(path) additionally rewrites single-brace {x} → {{x}} (without
touching an existing {{…}} pair); only the OpenAPI/Swagger paths call it, because in
Postman and Insomnia a single brace is literal text (/tags/{beta}, fields=friends{name})
and rewriting it invents a variable reference that resolves to nothing. Nunjucks tags
{% … %} and filtered vars {{ x | filter }} are left verbatim - Vayu has no equivalent
and renders them as literal text.
Dynamic variables in imported collections¶
Postman and Bruno collections routinely contain {{$guid}}, {{$timestamp}}
and the {{$random*}} faker names. The parse passes them through untouched -
normalize_template_vars only trims and strips a _. prefix - and they now resolve
at send time from the generator table described in
variable resolution. Before that
table existed they resolved to an empty string, so an imported request whose
body carried a {{$guid}} sent an empty field and the import looked successful.
A $name outside the supported set is left written as {{$name}} in the
outgoing request rather than emptied, so an imported collection using a faker
value Vayu does not have shows it instead of hiding it.
Sampler::sample¶
Sampler::sample(schema) - generates a sample value for an OpenAPI/Swagger schema,
used to build request-body stubs. It is bounded and resilient, not a naive one-level walk:
- Recurses up to
MAX_DEPTH = 6. - Resolves
$refvia the injectedresolveRef, with a per-pathSetcycle guard (a re-seen ref →{}); a failed/nullresolution →{}. - Returns a pinned value verbatim,
const→example→examples[0].constwins because JSON Schema makes it the only permitted value;examplesis OpenAPI 3.1's plural form. - For
allOf/oneOf/anyOf, walks the first branch (precedenceallOf → oneOf → anyOf). - Samples a 3.1 type array (
type: ["string", "null"]) from its first non-"null"member. - Type defaults:
string→""(orenum[0]),integer/number→0,boolean→false,null→null,array→[sample(items)](or[]),object/untyped → expandspropertiesrecursively (else{}).
schemaFormFields(schema, resolveRef) in the same module returns the sampled stub's own keys
([] when it samples to a non-object), each flagged file or not. It is how the v3 parser
reads urlencoded / multipart field names, so a form schema behind $ref or allOf resolves
as far as a JSON body does instead of reading a properties key that isn't there. The file
flag comes from the property schema (format: binary, or an array of it) rather than the
sampled value, which flattens a binary string to "" and cannot be told from a text field.
(the sampler, engine/src/core/openapi_drafts.cpp)
Adding a new format¶
All of it is engine-side (engine/src/core/import_document.cpp); nothing in
app/src needs to change, and the dialog picks the new format up for free.
- Write the parse in
import_document.cpp, emitting the draft model above (no ids, no persistence). Reuse what is there -map_key_values,to_var_record,imported_file_part,with_required_content_type, the OAuth mappers - rather than a second copy of a rule; the JavaScript semantics the drafts are held to (js_json.hpp) are shared with the OpenAPI builder for the same reason. A format that is not JSON or YAML (JMeter's.jmxis XML) cannot go throughcore::read_documentat all and needs its own reader and its ownis_<format>_documentraw-text check ahead of it inparse_import, in its own file (jmeter_import.cppbesideopenapi_document.cpp's own YAML reader) - see jmeter.md for the shape that took. - Add its detector to
parse_import's chain at the correct most-specific-first position, so it does not shadow (or get shadowed by) another format. - Populate
meta.skipped/meta.nonExecutableAuthfor anything Vayu cannot execute or represent, and letunattached_file_partscount uploads the user still has to attach, so the Preview can warn them. A loss the preview never names is the defect this whole subsystem is organised around. - Add cases to
engine/tests/fixtures/import-conformance.jsonand register the suite the wayengine/tests/CMakeLists.txtrequires. - Add a
docs/app/import-collections/<format>.mdfollowing the structure of the existing per-format docs, and a row in the table at the top of this file.