Category: Thoughts

Transforming Lidar Point Clouds

I was playing with datasets and wanted to share with what beautiful visualizations but wrong map alignment looks like.

The different colors represent scans but merged improperly (not aligned).

Looks like a mess.

But overall the coloring is important with debugging what this data represents.

Here’s what it looks like all merged together properly.

Especially the truck

Lidar map merging is a difficult task, and merging the data takes quite some time. So it is better to sample parts of a large data set and see if the algorithm is doing the right job.

At least the lidar above were all in the same 2D plane.

Here’s what it looks like if they weren’t — a blob of mess.

double roll = rtkMessage.roll();
double pitch = rtkMessage.pitch();
double yaw = – rtkMessage.heading();
double roll = deg2rad(rtkMessage.roll());
double pitch = deg2rad(rtkMessage.pitch());
double yaw = deg2rad(rtkMessage.heading());
double roll = deg2rad(rtkMessage.roll());
double pitch = deg2rad(rtkMessage.pitch());
double yaw = – deg2rad(rtkMessage.heading());

I tried a bunch of different methods and it turns out that I was using degrees in place of radians.

Here’s what after 5 days of working on this problem I finally solved

Here’s what the Google Maps view looks like

And here’s what the lidar point cloud data only looks like

Different view

Construction crane can be seen here.

radians roll/pitch

Very interesting in the map stitch process!

Compiling PDAL

sudo apt install libgdal-dev
git clone https://github.com/PDAL/PDAL.git
cd PDAL
mkdir build
cd build
cmake ..
make -j
sudo make install

List cmake options with installing this GUI

sudo apt-get install cmake-curses-gui ccmake ..
ccmake ..

To get geowave plugin support, you need JDK. Geowave increases the project’s complexity so I’ll defer that to later, if I get to it.

sudo apt install openjdk-8-jdk

Never Again, BTRFS! NAS File System Rant.

Synology DS918+ NAS

I have a Synology DS918+ NAS running BTRFS. This is 2019 and development of BTRFS began in 2007. So it should be all good, right?

There are interesting features I wanted to try out. Mainly, keeping snapshots. I’ve also been using EXT3/EXT4, NTFS, HFS, APFS, FAT32, even reiserFS back in 2006-2010, so I’ve seen my share of file systems. I’m willing to try something new.

I’ve heard 2 of my friends complain about this file system, but decided to try it because Synology’s Setup UI made me inclined to think this is preferable.

YouTube

I don’t know who the target audience for this web UI is, but recommending the BTRFS is a huge mistake. As a home user with a lot of pictures, it is just so unacceptably slow.

How slow is it?

I don’t have exact numbers, because they often time out. And I don’t want to go through the pain of waiting. I have about 16k jpeg files that in a flat folder. Simple operations are terribly slow.

  • Moving 100 files into a folder takes couple of minutes, because there is a copy made for each file moved.
  • Doing ls (list all files in folder) takes forever, my terminal hangs up and never returns even after hours.
    • On the Synology Drive UI, it takes minutes, because the web UI is paginated, about 500 files listed per page.
  • Deleting files takes as long as moving them.
  • Defragging like 1995 is back, so you must remember to periodically “scrub” your disks, which take days (I have two 4TB drives).

I regret so much using this shitty BTRFS system. Unbelievable. RedHat abandoned using this for all the right reasons, it’s hopeless. I’ll be spending a weekend moving my data to USB disks and reformatting to EXT4.

Never again, BTRFS!

XDinner

Wrapping up a project at and moving the project to phase 2. We celebrated with a team dinner at Cascal. Actually the first time our small project had outside dinner.

I didn’t take notes at the time so these are the memories from last night.

I will roughly summarize the events and some principles. I won’t write much into details partly for sake of privacy and partly because I can’t remember nor have the time to flush it all out. So I’ll write some memory triggers.

I had a bunch of questions in revolving around:
– What is going on with the project?
– What are the plans for our other projects?
– What were the lessons learned from building a company?

And other personal and life questions as
– Why did you create the company?
– How come you don’t want to be a professor anymore?
– Why did you want to become a professor?
– How did you climb the life ladder from where you started?
– What are your principles?
– How did you get to where you are today?

And some summaries of his responses
– work very very hard
– be opportunistic, they come and don’t wait
– surround yourself with people you want to become

Server migration

I’m in the middle of migrating the this website from hosted on a shared namecheap server to a VPS server. Namecheap notified me that this site will be undergoing for hardware upgrade, moving to a different physical server. Let me remember what the old hardware was, and I’ll update this post later with the new hardware. Schedule update is for Mar. 25, 2017.

[phepha@server159 ~]$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
stepping : 4
microcode : 1064
cpu MHz : 2099.884
cache size : 15360 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
…

[phepha@server159 ~]$ cat /proc/meminfo
MemTotal: 65786112 kB
MemFree: 740052 kB
Buffers: 10627716 kB
Cached: 38361744 kB
SwapCached: 0 kB
MemCommitted: 1031798784 kB
VirtualSwap: 0 kB
Active: 29757552 kB
Inactive: 25222428 kB
Active(anon): 4747632 kB
Inactive(anon): 1283136 kB
Active(file): 25009920 kB
Inactive(file): 23939292 kB
Unevictable: 5244 kB
…

[phepha@server159 ~]$ cat /proc/version
Linux version 2.6.32-604.30.3.lve1.3.63.el6.x86_64 (mockbuild@build.cloudlinux.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC) ) #1 SMP Sun Sep 27 06:34:10 EDT 2015