Python — It’s pretty slow

What are the alternatives?

If you’re a data scientist and need to analyze loads of CSV files for insights into, say, stock-price and market movements, the Julia programming language trumps machine-learning rivals Python and R, according to Julia supporters.

Some languages such as Rust aren’t widely used by developers but they are appreciated by programmers for qualities that excel in systems programming, versus application programming. For example, Microsoft is looking to Rust for the memory-safety features lacking in C and C++, which are extensively employed in Windows and other Microsoft projects.

Julia on the other hand has been adopted by some programmers for its C-like speed, but it has a much smaller ecosystem of packages than Python.

When working with a dataset that was a couple of gigs (HM Land registry Price Paid) as part of a take home challenge for a position I was applying for I created some functions that didn’t utilize the speed advantages of Numpy and Pandas to their full advantage and this resulted in very long processing times.

Rust is supposed to be a very fast language but it looks like it doesn’t have a large ecosystem of packages compared to Python or even Julia. My first step in increasing the speed is looking at how I can optimize Python code using either Cython libraries or just better use of Numpy and Pandas, and then start to learn more baout Julia and hope the ecosystem for it expands quickly.