[PWGHF] Reserve memory before filling charm femto tables

[fgrosa]

@zhangbiao-phy this PR adds the reservation of the memory before filling the tables, to prevent the following error:

[2492830:hf-producer-charm-hadrons-track-femto-dream]: [18:40:17][FATAL] Table 'FDHfCand3Prong': filled 37 rows after reserve(0). UnsafeAppend overran the reserved buffer — reserve() must request at least as many rows as are filled.

To to the change to the unsafe cursor introduced in AliceO2Group/AliceO2@6a773be

[github-actions]

O2 linter results: ❌ 0 errors, ⚠️ 1 warnings, 🔕 0 disabled

[zhangbiao-phy]

@zhangbiao-phy this PR adds the reservation of the memory before filling the tables, to prevent the following error:

[2492830:hf-producer-charm-hadrons-track-femto-dream]: [18:40:17][FATAL] Table 'FDHfCand3Prong': filled 37 rows after reserve(0). UnsafeAppend overran the reserved buffer — reserve() must request at least as many rows as are filled.

To to the change to the unsafe cursor introduced in AliceO2Group/AliceO2@6a773be

Hi @fgrosa, thanks a lot! I tried to fix that error by removing the reserve(), as you can see: #16637. It doesn;t work?

[fgrosa]

@zhangbiao-phy this PR adds the reservation of the memory before filling the tables, to prevent the following error:

[2492830:hf-producer-charm-hadrons-track-femto-dream]: [18:40:17][FATAL] Table 'FDHfCand3Prong': filled 37 rows after reserve(0). UnsafeAppend overran the reserved buffer — reserve() must request at least as many rows as are filled.

To to the change to the unsafe cursor introduced in AliceO2Group/AliceO2@6a773be

Hi @fgrosa, thanks a lot! I tried to fix that error by removing the reserve(), as you can see: #16637. It doesn;t work?

Thanks! I didn't notice it!
In principle reserving the memory would improve the performance, as if we do not reserve it, the safe cursor is used (so it checks the allocated memory every time it fills it, and if not enough, it allocates more). However I see that I basically re-implemented how it was before, but it was crashing, so better if I convert it to draft and I test more locally and I find the proper way to reserve the memory.

[zhangbiao-phy]

@zhangbiao-phy this PR adds the reservation of the memory before filling the tables, to prevent the following error:

[2492830:hf-producer-charm-hadrons-track-femto-dream]: [18:40:17][FATAL] Table 'FDHfCand3Prong': filled 37 rows after reserve(0). UnsafeAppend overran the reserved buffer — reserve() must request at least as many rows as are filled.

To to the change to the unsafe cursor introduced in AliceO2Group/AliceO2@6a773be

Hi @fgrosa, thanks a lot! I tried to fix that error by removing the reserve(), as you can see: #16637. It doesn;t work?

Thanks! I didn't notice it! In principle reserving the memory would improve the performance, as if we do not reserve it, the safe cursor is used (so it checks the allocated memory every time it fills it, and if not enough, it allocates more). However I see that I basically re-implemented how it was before, but it was crashing, so better if I convert it to draft and I test more locally and I find the proper way to reserve the memory.

I see, I just remove all the reseve(), but it's not the best way as you said. I am also trying to find a better way to fix it. then as Vit's suggestion, maybe we can fix it like EMCal did (https://github.com/AliceO2Group/O2Physics/pull/16580/changes). Reserved it like: rowCandCharm2Prong.reserve(rowCandCharm2Prong.lastIndex() + 1 + candidates.size());

[fgrosa]

@zhangbiao-phy this PR adds the reservation of the memory before filling the tables, to prevent the following error:

[2492830:hf-producer-charm-hadrons-track-femto-dream]: [18:40:17][FATAL] Table 'FDHfCand3Prong': filled 37 rows after reserve(0). UnsafeAppend overran the reserved buffer — reserve() must request at least as many rows as are filled.

To to the change to the unsafe cursor introduced in AliceO2Group/AliceO2@6a773be

Hi @fgrosa, thanks a lot! I tried to fix that error by removing the reserve(), as you can see: #16637. It doesn;t work?

Thanks! I didn't notice it! In principle reserving the memory would improve the performance, as if we do not reserve it, the safe cursor is used (so it checks the allocated memory every time it fills it, and if not enough, it allocates more). However I see that I basically re-implemented how it was before, but it was crashing, so better if I convert it to draft and I test more locally and I find the proper way to reserve the memory.

I see, I just remove all the reseve(), but it's not the best way as you said. I am also trying to find a better way to fix it. then as Vit's suggestion, maybe we can fix it like EMCal did (https://github.com/AliceO2Group/O2Physics/pull/16580/changes). Reserved it like: rowCandCharm2Prong.reserve(rowCandCharm2Prong.lastIndex() + 1 + candidates.size());

Indeed, even if in our case candidates.size() should be the maximum number of candidates (as they are not sliced per collision). I will test locally and see if I can find a good solution, thanks!

[fgrosa]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)

[vkucera]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)

Hi @fgrosa , thanks for the effort. Doesn't lastIndex return the number of rows - 1 in the current DF?

[fgrosa]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)

Hi @fgrosa , thanks for the effort. Doesn't lastIndex return the number of rows - 1 in the current DF?

Hi @vkucera that's right! I missed that we were consuming the collision iterator in the process function, so it is not necessary for the previous DFs, but due to the fact that we process collision by collision (so the candidate size is the one sliced per collision, and we have to account for the previous collisions). It should not be the case in the tree creators if we do not process candidates per collision (and also it was not in 05954fc, so I reverted it)

For a better understanding of what happens, here the log:

[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:05][INFO] Last index -1 candidate size 2, reserved memory 2
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:05][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:05][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 1 candidate size 8, reserved memory 10
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 2
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 3
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 4
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 5
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 6
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 7
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 9 candidate size 1, reserved memory 11
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 10 candidate size 1, reserved memory 12
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 11 candidate size 1, reserved memory 13
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 12 candidate size 1, reserved memory 14
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 13 candidate size 1, reserved memory 15
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 14 candidate size 1, reserved memory 16
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 15 candidate size 2, reserved memory 18
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 17 candidate size 1, reserved memory 19
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 18 candidate size 1, reserved memory 20
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 19 candidate size 3, reserved memory 23
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 2
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 22 candidate size 1, reserved memory 24
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 23 candidate size 1, reserved memory 25
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 24 candidate size 1, reserved memory 26
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 25 candidate size 1, reserved memory 27
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 26 candidate size 1, reserved memory 28
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 27 candidate size 3, reserved memory 31
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 2
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 30 candidate size 3, reserved memory 34
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 1
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 2
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 33 candidate size 1, reserved memory 35
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 34 candidate size 1, reserved memory 36
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Last index 35 candidate size 1, reserved memory 37
[3063536:hf-producer-charm-hadrons-track-femto-dream]: [10:20:06][INFO] Filling candidate number 0

[zhangbiao-phy]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)

many thanks @fgrosa! BTW could you also restore it for Charm + V0 producer, since I also remove them in this PR: #16637. Thanks again!

[fgrosa]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)
many thanks @fgrosa! BTW could you also restore it for Charm + V0 producer, since I also remove them in this PR: #16637. Thanks again!

Indeed I just did it here cb0a91c :) I also noticed that for Lc and D0 we might fill the table twice per candidate due to the two mass hypotheses, so I had to double the reserved memory.

[zhangbiao-phy]

Hi @zhangbiao-phy the implementation in these commits 6a34853 05954fc works (tested locally). Maybe this could be suggested also for the tree creators (+1 is not sufficient, as the size of the candidate table accounts only for the DF being analysed, while we should account for the previous DFs too)
many thanks @fgrosa! BTW could you also restore it for Charm + V0 producer, since I also remove them in this PR: #16637. Thanks again!

Indeed I just did it here cb0a91c :) I also noticed that for Lc and D0 we might fill the table twice per candidate due to the two mass hypotheses, so I had to double the reserved memory.

Thanks! I also want to say the same thing related to the mass hypothesis!

[alibuild]

Auto-approving on behalf of @fgrosa.