0.27.0 - The one with a better auto-transform mode

[justinvdm]

Since 0.19.0, we've been working on support different transform modes for snaplet:

• --transform-mode=strict: snaplet will fail creation of the snapshot and tell you which fields are missing from transform.ts
• --transform-mode=auto: snapshot tries to automatically transform the data for these fields, currently via copycat.scramble()
• --transform-mode=unsafe: snaplet will simply use the production data as is for these fields (the current default behaviour of snaplet)

While this feature is still experimental, we're getting close to having it ready for you use. This release include improvements towards this goal, specifically for --transform-mode=auto ("auto-transform mode"). Some of them might even help outside of just this feature.

Copycat 0.13.0

Since auto-transform mode relies heavily on copycat.scramble(), we need it to work as fast as possible. So we've made some improvements to copycat to make it faster and more secure, for scramble(), and more generally too.

You can read more about these changes here: supabase-community/copycat#36.

note: For any input given to copycat (for e.g. a column value in each row), the result will be different to what it was in previous releases.

Auto-transforming enums

In this release, we now support enum column types in auto-transform mode. When using snaplet in this mode, if snaplet sees an enum column for which a replacement value has not been defined in transform.ts, it'll choose from the available values for that enum. This is done deterministically - each row's

Truncating large values

In auto-transform mode, if snaplet sees a string value longer than 100 characters, before trying to scramble the value, it first truncates it to 100 characters.

Other improvements

Opting out of parsing json columns

Currently, snaplet parses json values into javascript objects, in order for us to pass it to your transform.ts config - this way you can work with the field as an ordinary javascript object.

// transform.ts
export const config = () => ({
  public: {
    SomeTable({ row }) {
      const { someJsonColumn } = row
      someJsonColumn.nestedValue += 1
      return { someJsonColumn }
    }
  }
})

In some cases though, it might be preferable to avoid snaplet doing this parsing, so that the raw json is given to your transform.ts instead:

// transform.ts
export const config = () => ({
  $options: {
    parseJson: false,
  },
  public: {
    SomeTable({ row }) {
      const { someJsonColumn } = row
      const parsed = JSON.parse(rawSomeJsonColumn)
      parsed.nestedValue += 1

      return {
        someJsonColumn: JSON.stringify(parsed)
      }
    }
  }
})

This can for example, be helpful to indicate to snaplet that you're wanting a json null and not a postgres null:

// transform.ts
export const config = () => ({
  $options: {
    parseJson: false,
  },
  public: {
    SomeTable() {
      return {
        someJsonColumn: 'null'
      }
    }
  }
})