My trip to the Moon… and Mars (part 2)

This article follows my previous post on my trip to NASA in Langley, Va. in early March 2019. This is the second and last part of this article. Before going […]

My trip to the Moon… and Mars (part 1)

I like to use π day, to remember a few things about science and technology that influence who I am. This year, π day is perfect for that. Let me tell […]

Spark in Action’s Chapter Eleven on Working with SQL is in MEAP

A new chapter of Spark in Action, 2e, (formerly known as Spark with Java) is available. Chapter 11 is titled “Working with SQL”. In chapter 11, you will explore how […]

(Almost) All you need to know about file ingestion in Apache Spark

As you may know, I start writing Apache Spark with Java (now renamed Spark in Action, 2nd edition). Usually, as the book develops, authors share a few excerpt of the book […]

Microsoft SQL Server 2019 gets a Spark

Yesterday, during Ignite 2018, Microsoft announced that they will integrate Apache Spark more tightly with SQL Server 2019. If you missed previous announcements around SQL Server, it now runs on […]

Lazy is good: understand why it’s good for you that Spark is lazy

This new chapter, chapter 4, of Spark with Java (https://www.manning.com/books/spark-with-java) is not only about celebrating laziness, it also teaches, through examples and experiments, the fundamental differences in building a data […]

The majestic dataframe in Apache Spark

Chapter 3 of Spark with Java is focusing on the dataframe. There is something majestic with Apache Spark’s dataframe, like those mountains of Montana. Apache Spark revolves around the concept of […]

Advanced Spark Ingestion

Chapter 9 still covers Spark ingestion (like chapter 7 and chapter 8), but this time, it’s about  “anything can become a Spark datasource.” When I was working for Zaloni, we […]

File Ingestion in Apache Spark

In a typical Big Data analytics scenario, you will probably be tempted to ingest files. You know, those pesky CSV files where the comma is sometimes a semicolon or a […]