Found in 5 comments on Hacker News
teleforce · 2025-10-23 · Original thread
According to ASN.1 Wikipedia entry, most of the tools supporting ASN.1 do the following:

1) parse the ASN.1 files, 2) generates the equivalent declaration in a programming language (like C or C++), 3) generates the encoding and decoding functions based on the previous declarations

All of these of exercise are apparently part of data engineering process or lifecycle [1].

Back in early 21st century Python is just another interpreted general purpose programming language alternative, not for web (PHP), not for command tool (TCL), not for system (C/C++), not for data wrangling (Perl), not for numerical (Matlab/Fortran), not for statistics (R).

D will probably follow similar trajectory of Python, but it really needs a special kind of killer application that will bring it to the fore.

I'm envisioning that real-time data streaming, processing and engineering can be D killer utility and defining moment that D is for data.

[1] Fundamentals of Data Engineering:

https://www.oreilly.com/library/view/fundamentals-of-data/97...

teleforce · 2025-08-30 · Original thread
For the foundation on data engineering I'd recommend this book by Joe Reis and Matt Housley. They did a good job on providing the framework that includes data engineering lifecycle, software engineering, data management, data architecture, etc. You can check the proposed framework here [1],[2].

[1] Fundamentals of Data Engineering:

https://www.oreilly.com/library/view/fundamentals-of-data/97...

[2] Fundamentals of Data Engineering Review:

https://maninekkalapudi.medium.com/fundamentals-of-data-engi...

teleforce · 2025-01-13 · Original thread
Not smart people but probably mostly software and computer people in general (but if you equate software and computer people with smart people then the statement is true).

Apparently, Fundamental of Data Engineering book does refer to cargo-cult metaphor inside its content [1].

[1] Fundamentals of Data Engineering:

https://www.oreilly.com/library/view/fundamentals-of-data/97...

teleforce · 2024-11-20 · Original thread
I highly recommend Fundamental of Data Engineering book [1].

Hopefully the authors can update the book soon to reflect the latest information and expand with another entire chapter for data management as they did to data architecture.

[1] Fundamentals of Data Engineering:

https://www.oreilly.com/library/view/fundamentals-of-data/97...

ludicity · 2023-11-25 · Original thread
I read something great to this effect recently, but I can't remember where. The gist of it is that adding people increases overhead, so the absolute smartest play is to keep the tiniest team physically possible and give them an absurd amount of leverage - which frequently takes the form of licensing the things that you really -can- commodify well.

The fundamentals of data engineering (https://www.oreilly.com/library/view/fundamentals-of-data/97...) is adamant on this. If you're a new data engineering team, just buy things that download the data you need, because fetching data from APIs is (usually) pretty simple to do with automated tooling. Then your team can focus on the part that we haven't nailed, like making sane models.