Automated historical data retrieval system for the Laboratorio Nacional de Observación de la Tierra

Main Article Content

Alejandro Aguilar Sierra
https://orcid.org/0000-0003-1018-8521

Abstract

After eight years of operation, the storage system at the Laboratorio Nacional de Observación de la Tierra reached a historical satellite data volume approaching one petabyte. To facilitate querying and data retrieval from this repository, a comprehensive infrastructure comprising three distributed components was designed and implemented: a web interface enabling users to submit detailed data requests, an asynchronous service for data extraction, and a provisioning service that creates temporary virtual accounts for downloading. The system was tested in real-world projects requiring a fine-grained selection of dates and times, successfully completing the retrieval process for a substantial data volume—890,000 files and 7.02 terabytes. This was achieved efficiently and with almost no human intervention, demonstrating the system's operational effectiveness and yielding valuable insights for future enhancements.

Downloads

Download data is not yet available.

Article Details

How to Cite
Aguilar Sierra, A. (2026). Automated historical data retrieval system for the Laboratorio Nacional de Observación de la Tierra. Cuadernos Técnicos Universitarios De La DGTIC, 4(4). https://doi.org/10.22201/dgtic.30618096e.2026.4.4.208
Section
Reportes técnicos
Author Biography

Alejandro Aguilar Sierra, Instituto de Geografía, UNAM

.

References

Abernathey, R. P., Blackmon-Luca, C. C., Crone, T. J., Henderson, N., Lepore, C., Augspurger, T., Banihirwe, A., Hamman, J. J., Gentemann, C. L., McCaie, T. A., Robinson, N. H., y Signell, R. P. (2021). Cloud-native repositories for big scientific data. Computing in Science & Engineering, 23(2), 26–35. https://doi.org/10.1109/MCSE.2021.3059437 DOI: https://doi.org/10.1109/MCSE.2021.3059437

Aguilar Sierra, A. (2020). Sistema de almacenamiento para el Laboratorio Nacional de Observación de la Tierra [tesis de especialidad, Universidad Nacional Autónoma de México]. TESIUNAM. https://tesiunam.dgb.unam.mx/cgi-bin/koha/opac-detail.pl?biblionumber=802512

Aguilar Sierra, A. (2025). Clúster de software libre del Laboratorio Nacional de Observación de la Tierra. Cuadernos Técnicos Universitarios de la DGTIC, 3(2), 24–34. https://doi.org/10.22201/dgtic.30618096e.2025.3.2.107 DOI: https://doi.org/10.22201/dgtic.30618096e.2025.3.2.107

Ansari, S., Del Greco, S., Kearns, E., Brown, O., Wilkins, S., Ramamurthy, M., Weber, J., May, R., Sundwall, J., Layton, J., Gold, A., Pasch, A., y Lakshmanan, V. (2018). Unlocking the potential of NEXRAD data through NOAA’s Big Data Partnership. Bulletin of the American Meteorological Society, 99(1), 189–204. https://doi.org/10.1175/BAMS-D-16-0021.1 DOI: https://doi.org/10.1175/BAMS-D-16-0021.1

Chard, K., Dart, E., Foster, I., Shifflett, D., Tuecke, S., y Williams, J. (2018). The Modern Research Data Portal: A design pattern for networked, data-intensive science. PeerJ Computer Science, 4, e144. https://doi.org/10.7717/peerj-cs.144 DOI: https://doi.org/10.7717/peerj-cs.144

Gomes, V. C. F., Queiroz, G. R., y Ferreira, K. R. (2020). An overview of platforms for big Earth observation data management and analysis. Remote Sensing, 12(8), 1253. https://doi.org/10.3390/rs12081253 DOI: https://doi.org/10.3390/rs12081253

Knapp, K. R., y Wilkins, S. L. (2018). Gridded Satellite (GridSat) GOES and CONUS data. Earth System Science Data, 10(3), 1417–1425. https://doi.org/10.5194/essd-10-1417-2018 DOI: https://doi.org/10.5194/essd-10-1417-2018

Kruskopf, M., Gilliland, L., Roberts, B., Markert, A., Ashmall, W., y Limaye, A. (2025). ClimateServ: An open source earth observation climate data access tool. Environmental Modelling & Software, 194, 106709. https://doi.org/10.1016/j.envsoft.2025.106709 DOI: https://doi.org/10.1016/j.envsoft.2025.106709

Mohr, M., Pebesma, E., Dries, J., Lippens, S., Janssen, B., Thiex, D., Milcinski, G., Schumacher, B., Briese, C., Claus, M., Jacob, A., Sacramento, P., & Griffiths, P. (2025). Federated and reusable processing of Earth observation data. Scientific Data, 12(1), 194. https://doi.org/10.1038/s41597-025-04513-y DOI: https://doi.org/10.1038/s41597-025-04513-y

Sudmanns, M., Tiede, D., Lang, S., Bergstedt, H., Trost, G., Augustin, H., Baraldi, A., y Blaschke, T. (2020). Big Earth data: Disruptive changes in Earth observation data management and analysis? International Journal of Digital Earth, 13(7), 832–850. https://doi.org/10.1080/17538947.2019.1585976 DOI: https://doi.org/10.1080/17538947.2019.1585976