YAML-ParseException/Malformed inline YAML string on start

[hbruch]

Observed behavior (please include a screenshot if possible)

When running datatools-ui with yarn run start, I often encounter YAML parse issues like this:

Error: Malformed inline YAML string or datatools-ui/node_modules/yamljs/lib/Parser.js:270 throw new ParseException('Unable to parse.', this.getRealCurrentLineNb() + 1, this.currentLine);

Logging the stacktrace yields something like:

at Parser.parse (./datatools-ui/node_modules/yamljs/lib/Parser.js:270:16)
    at Function.Yaml.parse (./datatools-ui/node_modules/yamljs/lib/Yaml.js:20:25)
    at DestroyableTransform._transform (./datatools-ui/node_modules/mastarm/lib/js-transform.js:92:47)
    at DestroyableTransform.Transform._read (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_transform.js:177:10)
    at DestroyableTransform.Transform._write (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_transform.js:164:83)
    at doWrite (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:409:139)
    at writeOrBuffer (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:398:5)
    at DestroyableTransform.Writable.write (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:307:11)
    at DuplexWrapper.ondata (./datatools-ui/node_modules/readable-stream/lib/_stream_readable.js:619:20)
    at DuplexWrapper.emit (events.js:198:13)
at Parser.parse (./datatools-ui/node_modules/yamljs/lib/Parser.js:270:10)
    at Function.Yaml.parse (./datatools-ui/node_modules/yamljs/lib/Yaml.js:20:25)
    at DestroyableTransform._transform (./datatools-ui/node_modules/mastarm/lib/js-transform.js:92:47)
    at DestroyableTransform.Transform._read (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_transform.js:177:10)
    at DestroyableTransform.Transform._write (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_transform.js:164:83)
    at doWrite (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:409:139)
    at writeOrBuffer (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:398:5)
    at DestroyableTransform.Writable.write (./datatools-ui/node_modules/mastarm/node_modules/readable-stream/lib/_stream_writable.js:307:11)
    at DuplexWrapper.ondata (./datatools-ui/node_modules/readable-stream/lib/_stream_readable.js:619:20)
    at DuplexWrapper.emit (events.js:198:13)

I suspect mastarm YAML parsing as culprit: the yaml file may be chunked and mastarm's js-transform seems to expect the whole file to be passed in one single call.

Expected behavior

No parse error should occure, as the YAML is correctly formatted.

Steps to reproduce the problem

Any special notes on configuration used

Version of datatools-ui and datatools-server if applicable (exact commit hash or branch name)

datatools-ui: b240cb6
datatools-server: v4.1.0

Perhaps there is a workaround to prevent this kind of issue? Otherwise, this would add another reason to replace mastarm (#819)

[miles-grant-ibigroup]

We have noticed this as well. Based on my testing it seems to be a race condition within mastarm. I've found that on node 14 it doesn't seem to cause a problem beyond printing the warnings.

I agree we should focus on removing mastarm. It's an arduous process but we're getting there.

[zoltan-mihalyi]

There is a function in mastarm/lib/js-transform.js, which is obviously wrong. If the file doesn't fit into a single buffer, it will try to parse each chunk separately.

function yamlTransform (filename) {
  if (!/\.yml|\.yaml$/i.test(filename)) {
    return through()
  }

  return through(function (buf, enc, next) {
    this.push(
      'module.exports=' + JSON.stringify(YAML.parse(buf.toString('utf8')))
    )
    next()
  })
}

[miles-grant-ibigroup]

Thanks for spotting this! We've done some work to try to repair mastarm in the past but it's difficult as it's such an old project. Thank you for your research, we'll be sure to try to get a PR in soon