It’s painful how much data rules the world

Standing in front of the convention center, next to the statue of Sir Walter Raleigh. On September 15th, 2021, after more than 18 months, I was finally able to give…

Bringing vision to Apache Spark

Drsti (pronounced drishti) is an effortless data visualization that interfaces easily with Apache Spark

Do your best work ever for Call for Code

I have been a mentor and judge for Call for Code. However, this year, I have other projects limiting my time to contribute to this world-changing initiative. That’s why I…

Building Enterprise Software Today

I am an enterprise architect. Among the things I work on, I am bridging technology and business at the enterprise level. When I am on the technology side, I am talking to a lot of engineers and architects of various levels. When I am on the business side, I am trying to explain our technology constraints. That’s why I wanted to level-set vocabulary and concepts that I considered critical. I have cut the content into three twenty-minute videos available on YouTube.

Get your own copy of Spark in Action 2e

Spark in Action, second edition is a favorite for the Big Bag Theory gang Spark in Action, second edition, has been out for about a month and was running a…

Hot July planning

Despite 2020 being a mess so far, and after a very calm period in terms of events, it’s time to get back on stage. July 2020 is going to be…

Awaited Apache Spark v3.0.0 is finally released

Apache Spark v3.0.0 hits the road, let’s celebrate! Apache Spark v3.0.0 has been released on June 18th, 2020, just before Spark + AI Summit 2020, which is being held virtually…

DataFriday: manipulating the schemas of Spark dataframes

A stretch… Data is organized by schemas, data is stored on disk (or memory), but nothing like a good old school disk. to illustrate data In this fifth episode of…

DataFriday: extracting metadata from photos

This Rolleiflex requires a physical piece of paper and pencil to store the photo’s metadata Following episode 3, where I talked about metadata in relational databases, this week, I am…