Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#IOPLT-325] Add Flatten Mapping and refactor enrichment #19

Merged
merged 1 commit into from
Jan 23, 2024

Conversation

AleDore
Copy link
Contributor

@AleDore AleDore commented Jan 22, 2024

List of Changes

  • add flattenField mapping
  • add flatten unit tests
  • Refactor blob enrichment
  • Refactor table enrichment

Motivation and Context

This is required to allow denormalization in specific use cases when data stream is composed by sub-objects that can be flattened. If flatten occurs field names are renamed if original name overlaps with other existing properties

How Has This Been Tested?

unit tests

Screenshots (if appropriate):

Types of changes

  • Chore (nothing changes by a user perspective)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@AleDore AleDore requested a review from a team as a code owner January 22, 2024 12:59
Copy link

Jira Pull Request Link

This Pull Request refers to the following Jira issue IOPLT-325

import { pipe } from "fp-ts/lib/function";
import { toJsonObject } from "../utils/data";

export const flattenField = <T>(input: T, fieldToFlat: keyof T): T =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export const flattenField = <T>(input: T, fieldToFlat: keyof T): T =>
export const flattenField = <T extends Record<string, unknown>>(input: T, fieldToFlat: keyof T): T =>

could be more explicit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used the same definition of other mappings :)
Maybe we can change generic type definition for all others formatters

@AleDore AleDore merged commit 9fe966e into main Jan 23, 2024
3 checks passed
@AleDore AleDore deleted the IOPLT-325_add_flatten_mapping branch January 23, 2024 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants