Save memory on collection
Notice that we do not need to fetch all items before we can push to the database (for example in sonarqube collector, we do not need to get all components before we can start upserting).
- Find api methods that go by the signature
GetAll...
, and evaluate if all the items are necessary to be in memory before upserting logic can take place. - If they do not all need to be in memory, change the api method to just return an item per page, and upsert each page items and store the record in a map for checking wether saved items need to be deleted
(Can use sonarqube collector as example).