Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support denormalising data in references #2252

Open
alcaeus opened this issue Nov 27, 2020 · 0 comments
Open

Support denormalising data in references #2252

alcaeus opened this issue Nov 27, 2020 · 0 comments
Labels

Comments

@alcaeus
Copy link
Member

alcaeus commented Nov 27, 2020

Feature Request

Q A
New Feature yes
RFC no
BC Break no

Summary

When storing references as object (e.g. by using dbref or ref strategies), we can add data to the reference. This is currently only used to store a discriminator field if the reference or target document uses discriminator mappings, but could be used for any other fields from the additional document. For example, one could want to denormalise a username in a user reference:

{
    "title": "Test article",
    "author": {
        "id": <...>,
        "username": "alcaeus"
    },
    <...>
}

Why would users want to do this?

  • Users could query articles by username without having to use the $lookup aggregation pipeline stage to filter by username
  • Proxy objects generated for this reference would initialise the proxy with the username from the reference, meaning that accessing only the username would not require proxy initialisation
  • This can be used to duplicate data for record-keeping purposes (think of storing product details with an order in an e-commerce platform).

One big issue in this whole thing is updating the stored value. In my initial proof of concept (see #1527), I mentioned that the denormalised value is not yet updated for two reasons:

  • Tracking incoming references: this is a very big issue that requires an overhaul of the metadata system. In the example above, the user document would have to know that there is an incoming reference from an article document that needs updating. That is not reliably the case at the moment (it can be done using inverse references but there are better ways). One way to solve this would be to introduce an object graph to track metadata: this would allow us to efficiently traverse the metadata graph to find incoming references and update these values (if so specified in the options).
  • Query support when updating nested values: at the time the proof of concept was created (a whopping 4 years ago), MongoDB did not support using multiple positional dollar operators in queries, limiting updating such references to non-nested fields. With array filters in update options added in MongoDB 3.6, we could take advantage of the $[<identifier>] operator to update such values when the original document is modified.

Both issues require extensive changes under the hood, especially to Metadata handling which is currently provided by the doctrine/persistence library. For that reason, I wouldn't want to have that as part of the first implementation, meaning that denormalised values are never updated. Updating will be an opt-in feature (due to potential performance implications). Due to potential inconsistencies, we should also hold off on building this opt-in feature until transaction support has been completed (which also requires large-scale changes under the hood).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant