Import function should be able to create paths with no keys, for proper migration

[umutkacar]

import function skips creating paths that have no keys and I think this behavior contradicts the purpose of migrating, because the source and target data structures may not be the same at all cases.

Example;

Here we have a structure where each directory has its own global directory. Some have keys under them, some others do not:

safe tree --keys secrets/data_science
.
└── secrets/data_science/
    ├── data_engineering/
    │   ├── debezium-backend/
    │   │   ├── global
    │   │   ├── production/
    │   │   │   └── global
    │   │   │       ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
    │   │   │       └── :JDBC_LOG_DWH_READONLY_PASSWORD
    │   │   └── staging/
    │   │       └── global
    │   │           ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
    │   │           └── :JDBC_LOG_DWH_READONLY_PASSWORD
    │   └── global
    └── global

We can also observe it with:

safe paths --keys secrets/data_science
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_READONLY_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_READONLY_PASSWORD

...where we see the paths with keys, and

safe paths secrets/data_science
secrets/data_science/data_engineering/debezium-backend/global
secrets/data_science/data_engineering/debezium-backend/production/global
secrets/data_science/data_engineering/debezium-backend/staging/global
secrets/data_science/data_engineering/global
secrets/data_science/global

...where we see all paths, with the globals that do not have keys.

When we export this path, the output actually has info about these empty paths, secrets/data_science/data_engineering/global and secrets/data_science/global:

safe export -a secrets/data_science | jq
{
  "secrets/data_science/data_engineering/debezium-backend/global": {},
  "secrets/data_science/data_engineering/debezium-backend/production/global": {
    "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
    "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
  },
  "secrets/data_science/data_engineering/debezium-backend/staging/global": {
    "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
    "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
  },
  "secrets/data_science/data_engineering/global": {},
  "secrets/data_science/global": {}
}

but the import function does not create these paths on the target Vault setup. Hence the target structure fails to resemble the source.

Here's the command I use to do a migration, with safe v.1.8.0, source_vault version 1.11.12, target_vault version 1.15.2:

safe -T source-vault export -a secrets/data_science | safe -T destination-vault import secrets/d
ata_science
wrote secrets/data_science/data_engineering/debezium-backend/global
wrote secrets/data_science/data_engineering/debezium-backend/production/global
wrote secrets/data_science/data_engineering/debezium-backend/staging/global
wrote secrets/data_science/data_engineering/global
wrote secrets/data_science/global

The migrated paths are missing the empty paths: secrets/data_science/data_engineering/global and secrets/data_science/global:

safe paths secrets/data_science
secrets/data_science/data_engineering/debezium-backend/production/global
secrets/data_science/data_engineering/debezium-backend/staging/global

...even though the keys are migrated intact:

safe paths --keys secrets/data_science
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_READONLY_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_READONLY_PASSWORD

I think we should have an option for the import function that supports creating these empty paths on the target Vault, since some application may depend on them and it would be a burden to create those paths manually on a big migration project.