7a18b1b2ff
API: started work, added a serializer
tyrin2024-09-17 11:37:03 +03:00
c6df41f903
supplementary to the previous
tyrin2024-09-16 15:47:02 +03:00
ea9832a365
Remove gaia designation from the model to save on disk space
tyrin2024-09-16 15:46:01 +03:00
3d9c950751
Remove most fields from the model to save disk space
tyrin2024-09-16 11:27:56 +03:00
c122b271e9
Indexer: made code more legible, added some comments
tyrin2024-09-16 10:44:40 +03:00
862efc5cbd
Indexer: additional work on the multithreaded version
tyrin2024-09-16 10:35:43 +03:00
eec4ebbbfc
Indexer: switched to user input (with previous value support) for the NSIDE value. Testing indexer_multithread version added. Ingester: minor code tweaks
tyrin2024-09-13 15:27:49 +03:00
ec27a495b1
Additional work on the Indexer
tyrin2024-09-13 12:19:04 +03:00
c27ac6ce97
Experimental fix to the memory leak issue in ingester. Additional work on the indexer
tyrin2024-09-13 10:43:33 +03:00
e5663e3c30
removed boolean types for ring and nested indices. Made the indexer create both indexes at once as it is faster.
tyrin2024-09-12 16:47:00 +03:00
84f373d590
added ORDER and NSIDE as arguments for the indexer script
tyrin2024-09-12 15:52:38 +03:00
2d45a653fc
added the current_time function to ingester too
tyrin2024-09-12 11:52:56 +03:00
4eaad008e1
added the monitor script cycle.sh. Made it check for the finish flag file. Added the finish flag generation to ingester.
tyrin2024-09-12 11:50:23 +03:00
e37387b68f
made ingester functional without inputs in preparation for a monitor script restarting it
tyrin2024-09-12 11:45:02 +03:00
2ea452cf81
returned to pyarrow backend in pandas, added timestamped outputs to the indexer implementation
tyrin2024-09-12 10:43:43 +03:00
0f3d33ac13
switched backend back to numpy so it maybe fixes the memory leak
tyrin2024-09-12 10:32:33 +03:00
a8854fe4a4
removed regular gc calls; added var=None in attempt to mitigate memory leaks
tyrin2024-09-12 09:46:31 +03:00
b4117d2ee7
implemented regular garbage collection in ingester
tyrin2024-09-11 15:45:46 +03:00