High Definition Mapping Infrastructure

High definition mapping (HDM) is a form of mapping that uses advanced technologies such as LiDAR, satellite imagery, aerial photographs and photogrammetry to create detailed, accurate 3D maps or models of an area. HDM is used for a variety of purposes, from creating virtual tours of a city to helping autonomous vehicles safely navigate roads.

First, I’d like to share a little about myself and the relevant experiences. I worked with high definition mapping.

Most recently I worked on developing a home robot (Astro) at Amazon Lab126, building out the embedded mapping and infrastructure framework.


Prior to that, I worked at AutoX — a self driving vehicle company, mostly a generalist systems engineer. I worked on building and scale out the infrastructure to handle massive amount of data.


In this article, we’ll focus primarily working with point cloud data in terms of infrastructure, compute, data structures, and some algorithms to compute and transform the data.

High Definition Maps

High Definition Maps contains 3D data, produced from include the raw point data (“point cloud”) as well as processed derivatives such as contours and surfaces (DEMs).

HD maps differ from conventional maps in that it is possible to use HD maps for high accuracy and precision localization (e.g. knowing where you are on the road on a centimeter level).

Continue reading

Domain-specific DNS server on MacOS

Check out your DNS configuration using $ scutil --dns . Resolver #1 is what is handling my DNS lookups, #2 handles the .local domain. Resolver #3 is the reverse lookup for the 169.254 APIPA address space, and so on.

jason@jmbp15-ati ~> scutil --dns                                                                                                                                                                                                          18:10:04
DNS configuration

resolver #1
  search domain[0] : home
  nameserver[0] :
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #8
  domain   : ds
  nameserver[0] :
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : home
  nameserver[0] :
  if_index : 4 (en0)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

Resolver #8 is what I have added to resolve a custom TLD “ds”.

To add an additional resolver to a Mac, create a directory at /etc/resolver.

sudo mkdir /etc/resolver

For each domain that you want to hit a specific nameserver, create a file with the name of your desired domain and a nameserver line (or lines) in the file. For my internal domain I used the following command:

echo '' > /etc/resolver/ds

Now, when I run scutil --dns again I see my newly created resolver:

resolver #8
  domain   : ds
  nameserver[0] :
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

Note; needs to be a DNS resolver.

A quick lookup confirms that my configuration is doing what I want it to do. Another thing I discovered when looking into this is that dig and nslookup on OSX don’t use the OS resolver configuration.

$ dscacheutil -q host -a name sanfran.ds
name: sanfran.ds

And that’s it. If I want to configure forward or reverse zones to resolving using a specific nameserver on OSX it’s that simple.

[C++] What is copy_elision and why you should know about it

In the C++ world, copy_elision is a compiler optimization.

Prior to C++11, there was no copy_elision, so when you write code like this…

Person makePerson() {
    Person p();
    p.name = "Bob";
    return p;

… the compiler creates a Person object in the makePerson stack frame and is copied/moved to the caller’s stack after makePerson finshes.

But with with copy_elision, the compiler optimizes the process and rather than a move/copy, compiler sets the address of makePerson’s Person directly to the return address. In other words, there is no move/copy.

Explained by the official website https://en.cppreference.com/w/cpp/language/copy_elision

The objects are constructed directly into the storage where they would otherwise be copied/moved to.

This optimization is usually unnoticed.

Great! So what? Why do I need to care about this?

Tl;dr: When you lie to the compiler.

As a general rule: if you lie to the compiler, there lies bugs, undefined behavior, and probably stack smashing.

Let me tell a story about using ROS.

ROS creates a layer of abstraction for inter process communication (IPC) with sockets, in a publisher/subscriber model.

The ROS semantics for a subscriber is very overloaded, as in there are many definitions of “subscribe”.


Here’s what happened.

I defined an Object and created a callback. E.g. Cards object and callback_add_card(Card newCard).

Important: The callback returned an int. E.g. “int callback_add_card”.
This leads me to make a mistake… We shall see why later.

I constructed the object and passed that object’s callback to ROS subscribe.

But ROS’s subscribe definition expected (void*) function pointer.
I.e. void callback_add_card was acceptable but int callback_add_card is not.
So I lied to the compiler and cast this to a void*. Subtle mistake. Won’t show up until later.

#define callback_add_card_fn (void (Cards::*)(const Card&))

nodeHandle.subscribe("/cards/add", subscribeQueueSize, callback_add_card_fn &Cards::callback_add_card, &my_cards);

The compiler was happy and compiled. Because the “#define cb_car_speed …” told the compiler to treat the function as void.

Code mostly worked fine.

Because an int is relatively small (byte), there appears to be no bugs because the stack smashing wasn’t obvious.

Sometime later, I changed the return type.
From int callback_add_card to Card callback_add_card.
Because why not just return the Card instead? So I can write some test code and see if the Card is correct.

Well! Now we have big trouble.

The compiler was told this callback_add_card had void return type. So compiler did not allocate any space for a return value. And because of copy_elison, the compiler had generated code for callback_add_card “directly into the storage where they would otherwise be copied/moved to.

In other words, calling callback_add_card smashes the stack of the caller. Because the caller expected void return value, and no spaces was allocated. But the copy_elison code constructed the Card object in the caller’s stack frame!

So how do we get around this? How to have a callback that returns some value, but still call the ROS subscribe properly?

Answer: By wrapping the call in a lambda, creating a closure. The lambda returns no value, so it’s a void function. But the lambda can capture context. E.g.

nodeHandle.subscribe<Card>("/cards/add", subscribeQueueSize, [&my_cards](Card newCard) {
  cout << "the card added is: " << my_cards.callback_add_card() << "\n";

Connect Bose QC35 to Ubuntu 16.04

1 /etc/bluetooth/main.conf

# Restricts all controllers to the specified transport. Default value
# is “dual”, i.e. both BR/EDR and LE enabled (when supported by the HW).
# Possible values: “dual”, “bredr”, “le”
ControllerMode = bredr

Restart bluethooth service afterwards: sudo /etc/init.d/bluetooth restart

After the pairing you may put the config back to “dual”.  Only pairing seems to be affected.

2 Headphones to pair mode

Slide button 2s or use their app (assuming already paired with your phone).

3 Settings -> Bluetooth -> +

Click on PIN options and select Fixed PIN Do not pair.

4 Connect again in side bar

5 Sound setting to High Fidelity

Meshroom 3D point cloud creation from photos

I went to a museum in SF today, and took some photos of Cosimo de Medici – an influential Italian. I just finished watching Medici.

Sadly not enough photos for Recap Photo (20 needed). Should’ve taken more.

Should’ve taken more photo of this too.

Very exciting – I could take photos and have a point cloud rendered.