20 Commits

Author SHA1 Message Date
7a18b1b2ff API: started work, added a serializer 2024-09-17 11:37:03 +03:00
c122b271e9 Indexer: made code more legible, added some comments 2024-09-16 10:44:40 +03:00
862efc5cbd Indexer: additional work on the multithreaded version 2024-09-16 10:35:43 +03:00
eec4ebbbfc Indexer: switched to user input (with previous value support) for the NSIDE value. Testing indexer_multithread version added. Ingester: minor code tweaks 2024-09-13 15:27:49 +03:00
e83b23a35c Indexer: improved outputs, added progress reporting. 2024-09-13 13:09:39 +03:00
ec27a495b1 Additional work on the Indexer 2024-09-13 12:19:04 +03:00
c27ac6ce97 Experimental fix to the memory leak issue in ingester. Additional work on the indexer 2024-09-13 10:43:33 +03:00
e5663e3c30 removed boolean types for ring and nested indices. Made the indexer create both indexes at once as it is faster. 2024-09-12 16:47:00 +03:00
84f373d590 added ORDER and NSIDE as arguments for the indexer script 2024-09-12 15:52:38 +03:00
3acef66a89 monitor.py to restart ingester periodically hopefully avoiding memory leaks. 2024-09-12 14:30:59 +03:00
83888cd2f9 removed script 2024-09-12 13:53:09 +03:00
2d45a653fc added the current_time function to ingester too 2024-09-12 11:52:56 +03:00
e37387b68f made ingester functional without inputs in preparation for a monitor script restarting it 2024-09-12 11:45:02 +03:00
2ea452cf81 returned to pyarrow backend in pandas, added timestamped outputs to the indexer implementation 2024-09-12 10:43:43 +03:00
0f3d33ac13 switched backend back to numpy so it maybe fixes the memory leak 2024-09-12 10:32:33 +03:00
a8854fe4a4 removed regular gc calls; added var=None in attempt to mitigate memory leaks 2024-09-12 09:46:31 +03:00
b4117d2ee7 implemented regular garbage collection in ingester 2024-09-11 15:45:46 +03:00
5fe96c3b86 prototype indexer implementation 2024-09-11 15:06:47 +03:00
6df8af77df started work on the indexer script 2024-09-11 12:11:33 +03:00
9f26052fe2 switched root dir to the application itself 2024-09-10 15:08:51 +03:00