Inside Neouart – Tricking a serial port into being a signal generator

Last time, we connected a NeoPixel directly to a RaspberryPi. This is certainly fun and useful, but the real motivation behind this project was to explore clever ways to make use of limited hardware resources. NeoPixels need a precisely timed string of bits to be happy. Luckily, every RaspPi comes with built-in hardware for generating strings of precisely timed bits – a serial port!

If all you care about is making pretty colors the easy way, don’t bother reading this article. If you are wondering how it is possible to reliably generate a pulse train with +/-150ns precision on a Raspberry Pi pin without kernel mods or DMA, then read on!

Continue reading

Give Your Raspberry Pi a NeoPixel

Everyone loves Raspberry Pi and everyone loves NeoPixels, so let’s get a NeoPixel directly under your Raspberry Pi’s control!


The Pi can set the NeoPixel to any RGB color and you can create scripts to update it up to 100 times per second.

For a few cents worth of parts (actually just one part!) and a single IO pin, you can give your Pi…

  • an animated multicolor status light
  • an i-style breathing power indicator
  • an adorable single pixel Ambilight back-light behind your tiny screen
  • a sunrise-synchronized night light

The possibilities are endless! Click on for a demo video and full instructions…

Continue reading

Save the Earth One Resistor at a Time – External pull-up resistor no longer needed for DS18B20 temp sensor


Using the updated Arduino 1-Wire library code presented here, you can eliminate the need for an external pull-up resistor for typical small networks of DS18B20 temperature sensors.  This should also work with any AVR processor and other types of 1-Wire devices as well. You can download the updated 1-Wire library here…


The mythical “required” external pull-up


From the DS18B20 Data Sheet

From the DS18B20 Data Sheet

If you’ve ever used the ubiquitous (and amazingly useful!)  DS18B20 family of 1-Wire temperature sensors, you’ve almost certainly used a 4.7K ohm pull-up resistor as well. Every one of the seemingly endless Arduino DS18B20  tutorials on the web starts with some version of the line “You will not be able to do anything with this senor until you go out and procure yourself a 4.7K ohm resistor”. AdaFruit is even generous enough to include one of these resistors with every DS18B20-based temperature sensor they sell (be it bare, waterproof, and hi-temp) because they know you are going to need it.

I am here to tell you that everything is about to change. If you were banking on your stockpiles of 4.7K ohm resistors to be the one reliable store of value in these uncertain times, you need to rethink your long-term asset preservation strategy because the decade-long run of stable demand for this part is about to plummet. Yes – it is now possible to connect DS18B20 sensors without any external pull-up resistor at all!

Outrageous claims demand outrageous proof, so let’s start with a brief demo that proves beyond a shadow of a doubt that this is not just a cockamamie theory, but cold hard fact…

Continue reading

NeoPixels Revealed: Warping time and space to actually see inter-pixel jitter

This is Part 5 in a series. Read Part 1 here.


In the PWM article, we discovered that each NeoPixel has a clock that free-runs at about 500Hrz and each pixel will only update its displayed color at the end of a clock cycle. This means that there should be about 2ms of jitter when updating a pixel, which absolutely limits the maximum refresh rate.

This sounds good in theory, but how can we actually see something that happens over the course of 1-2ms?

My first strategy was to haul out the amazing Nikon 1 J4 which can shoot at an astonishing 1,200 frames per second. It is no Phantom, but at 0.8ms per frame it is just good enough to catch some of the jitter. Here is a video of a string of NeoPixels blinking off and on a couple of times over the course of 0.05 seconds…

The amber indicator LED on the far left lights up immediately after the latching reset pulse is sent, then for the next couple of frames you see the Neopixels individually update at random times according to the phase of their internal PWM clocks. It takes a full 3 frames (9-12) which corresponds to about 2.4ms (3 frames *0.8ms/frame) for all the pixels in this string to update.

While this is really cool and seems to confirm that the pixel jitter really exists in the real world, we are sampling at a rate very close to the rate of the thing we are trying to measure which could mislead us.  If only we could get more temporal resolution. How can we see down to microseconds without spending $20k on a fancy camera?

Hmm… My camera has *lots* of spatial resolution (5232 x 3488!), and we only even really need one of those two spatial dimensions (width) to see our linear string of NeoPixels. If only we could trade the other spatial dimension that we are not using (height) for temporal resolution… but how can we trade space for time?

All it takes is tripod with a tiltable head and a quick wrist! Here is the setup…

time warp setup


If we point the camera at the pixels and then quickly scan the camera’s field of view across the pixels, we will record their time transitions as changes along the y-axis of our photo! Here was my very first attempt at dimensional arbitrage…

First try at time warp photo

I think this image is pretty freaking beautiful! It very clearly shows the LED jitter with a resolution on the order of sub-milliseconds. It even captures the fact that I was only actually blinking the leftmost 42 pixels – the rest just happened to be 50% white and blinking at the PWM frequency of about 400hz. To the naked eye, this string just looks like a slightly dimmed white line, but with this time warp technique we can see into the secret high-speed world. I can hear the Scalosians buzzing!

This trick is nothing new. It is the same thing going on with the now trendy light painting displays and I’ve even been doing it since the 90’s. But it is still an amazing way to turn a $100 camera into a $20K high speed recorder – as long as you don’t mind loosing a dimension.

After some practice to get my tripod-tilting fast and steady and also turning out the lights in the room, I was able to get this close up shot…


The amber indicator LED on the left is on for just 125 nanoseconds (2 CPU cycles at 16Mhz). It marks the end of the latching reset pulse that tells the NeoPixels to update their displays. You can very clearly see the displays each update when their PWM cycles finish. Keep in mind that the camera was tilting from down to up, so time runs from top to bottom. If you look really closely, you can even see the very brief 120us off period that happens between PWM cycles of the pixels that are on for more than one cycle. Try clicking on the photo to zoom. It looks like the necking of a string of hot dogs. See? Just by counting, we can see that some of the pixels are on for 1 PWM cycle, while others are on for 2, and a couple that hit it just right are even on for 3 cycles- all depending on what phase their PWM cycle happened to be in when the latch pulse arrived.

Ok, enough shop talk. Let’s now look at a pretty NeoPixel time warp picture!…



This photo shows the string cycling though full brightness red, green, blue, and white. To the naked eye this also just looks like a solid white bar – but if you happen to shift your gaze quickly you can just catch the colors out of the corner of your eye as they paint across your retina. It is a neat effect because the colors disappear again as soon as you look at them.



Q: How fast can you go?
A: The limits on time resolution include

  1. The vertical resolution of the camera,
  2. How much light hits the image recorder,
  3. How fast you can tilt the camera.

My camera has 5000+ pixels in the Y axis and the NeoPixels are really bright, so the limiting factor for me is how fast I can tilt. I’ve gotten much higher resolution by putting the camera far away from the pixels and zooming in. This increases the effective speed. If you wanted to go even faster, you could use a spinning mirror instead.

Q: How long was the strip powered up for before you did the test? I bet the pixels started of all in sync, but drifted over time.
A; I actually thought this too, and so did a test about 1-2 second after repowering the string. The results are the same. I should have guessed this because in the first image, you can see that even in the short instant of the exposure the drift between pixels is noticeable otherwise the dot pattern on the right would be smooth.

Temperature Logging to a Google Spreadsheet with an Arduino Yun

UPDATE 8/7/2104:

It appears that Google occasionally will randomly and silently drop the authorization for an Apps Script web app. If you notice that your spreadsheet has stopped updating and you know that your logger is still working, then you probably need to log into your spreadsheet from a web browser, go into the scripts editor, and manually execute any function in the script. This will cause a popup that will reauthorize the script and everything will then start working again.

I think this is the last straw. I can not recommend using Google for logging (or any other non-trivial application) any more. This stuff is just too flaky. Sorry.

If you are like me, you often find yourself in a situation where you need to log multiple channels of temperature data. From testing nano-insulating paint to debugging an overheating geothermal well, most of us will have need of accurate and frequent temperature logs at some point in our short and brutish lives.

Here is a recipe to make a reliable, cheap, and easy cloud-based, multi-channel temperature logger using an Arduino Yun and DS18B20 temperature sensors. I chose the Yun because everyone loves Arduinos and this one can connect to the internets. I choose the DS18B20 sensors because they are awesome and cheap and accurate and you can hook up lots of them to a single pair of wires.

We will be using a Google Spreadsheet to store our logged temperature data for a wide array of reasons detailed in my previous previous article.

Quick Start

  1. Get a Yun and plug it into power and the internet. I use the Ethernet port because it is easy, but it should work fine if you connect over Wifi.
  2. Connect a DS18B20 sensor (or lots of them!). The middle pinis data and goes to to pin11 on the Yun and the other two pins (power and ground) can go to any of the pins labeled “GND” on the Arduino. (You don’t even need a pull-up resistor for just a couple of sensors!)
  3. Install the Dallas Temp Library and 1-Wire library into your Arduino IDE if you don’t already have them.
  4. Load up the SimpleTempLoggerYun code into your Yun using the Arduino IDE.
  5. Check the example sheet here and you should see your data show up a few seconds after the Yun boots!
  6. When you are ready, go ahead and make your own exclusive logging spreadsheet and then copy/paste the URL into the SimpleTempLoggerYun code (near the top where it says GOOGLE_URL) and download that new code into your Yun.
  7. If you are happy and plan on using you new logger for a while, you will need to fix this OpenWRT bug on your Yun.

Done! Now make some cool live graphs and share them with your friends.


There are some very nice things about using DS18B20 sensors.

  • They are very cheap. I got 10 waterproof sensors with wire harnesses for $2 each including postage. How is that possible? Loose parts are as cheap as $1.50.
  • They only take 2 wires to connect – that’s only a single digital pin.
  • You can hook them up in pretty much any configuration you need.
  • They are amazingly responsive. I can tell within 5 seconds when someone opens the door to my roof because of the detected temperature change.

Each sensor has a data line, a power line, and a ground line. All you care about is knowing which one is data – you will connect the other two together anyway.

Here is what the loose part looks like….


The middle pin is the data. I used to think that soldering wire directly to the pins was the fastest and easiest way to get these connected…


…but after doing a lot of them this way, I finally figured out that is is *much* easier to use a board….


AdaFruit is nice enough to send me free boards every time I order something, so I cut these boards up into little pieces to make DS18B20 carriers. It seems like more work, but it is so much easier in the end.

Note that in the first photo and I using crappy Radio Shack speaker cable, but in the second I am using a single pair of a CAT5 cable here. Another example of live and learn. Speaker wire works if you only have a couple sensors or a couple meters of cable, but CAT5 works with dozens of sensors and dozens of meters of cable. Here is an experiment that used a string of 12 sensors connected to about 20 meters of CAT5 wire and it worked great….


Here is what the waterproof packaging looks like…


At first I was worried that the extra metal on the end would raise the thermal mass and slow down the response to temperature changes, but these turned out to still be plenty fast and can easily register the heat of you touching one with a finger in 1 second.

The water proof ones are very handy because they already attached wire. On the ones I get from Amazon, the data pin is the white wire. On the AdaFruit version, the data wire is yellow.

I typically solder a bunch of them to a board fragment in a star configuration like this…


I recently saw this terminal block on Amazon that could make hookup solder-free.

I’ve also learned that you really should connect the power pin to ground if are going to use only two wires (parasite power mode). They will work if you leave the power pin dangling, but not as reliably probably because the unconnected power pin picks up noise.  For me, this manifested itself in occasional readings of 185C, which indicates that the sensor reset between the time you told it to take a sample and when you tried to read that sample over the bus. If you are getting 185C readings, maybe check that your power pins are solidly grounded.

Note that the wire used on the waterproof sensors is not twisted pair and so you will be limited in how much wire and how many sensors you can hook up before it starts getting flaky. I’ve found that I can reliably use about 2 waterproof sensors without a pull-up resistor, and about 6 with one- especially if the wires from the sensor are connected right near the board so there are not extra runs.


Problems along the Way

It should not be hard to hook up a Yun as a Cloud data logger – that’s what it is built for, right?

But it was hard. Too hard.

Problem #1: Yun looses internet connection after working for 12 hours.

Looking at the logs, curl is failing with “Failed to connect: network unreachable” message. Netstat -r shows the default gateway is just gone.

This turns out to be an outright bug in the scripts on the Yun that make it delete the default gateway anytime it does a DHCP refresh, which happens for most networks every 12 or 24 hours.

You can read about the bug here and how to fix it here (thanks Mitu!)…

Moral: Fix that script on any Yun you plan to leave connected to the internet.

Problem #2:  Logger works perfectly on my desk while I am watching it, but fails after I leave.

I can sit there and watch it all day and it keeps on chugging perfectly.

Alas, anytime I leave it alone for more than about 30 minutes, it starts dumping garbage data into the spreadsheet. In an attempt to catch the glitch in the act, I went for a walk but kept watching the spreadsheet for garbage data. As soon as I saw the first bad row, I rushed back to my desk and watched for the next line of garbage data to go out- but the next line was fine. No matter what I did the data was good if I was sitting there watching it and would turn bad about 30 minutes after I left. It felt like a prank, but there was a simple explanation…

Continue reading

Using Google Spreadsheets for Logging Sensor Data

UPDATE 8/7/2104:

It appears that Google occasionally will randomly and silently drop the authorization for an Apps Script web app. If you notice that your spreadsheet has stopped updating and you know that your logger is still working, then you probably need to log into your spreadsheet from a web browser, go into the scripts editor, and manually execute any function in the script. This will cause a popup that will reauthorize the script and everything will then start working again.

I think this is the last straw. I can not recommend using Google for logging (or any other non-trivial application) any more. This stuff is just too flaky. Sorry.


You want to…

  • remotely monitor the temperature in your greenhouse for the past 24 hours, but only have a dynamic IP address on premises
  • keep a historical record of 1,500 wind readings readings per hour from net-connected anemometers at your favorite surfing spots
  • create an earthquake monitoring board that you can distribute to 1,000’s of people and have all the boards report back any earthquakes to a central data store that everyone can see and run analysis on in real-time

Now that Google Spreadsheets can handle up to 2 million cells, they are a great place to log all your sensor data. You can easily view the live data from any computer on the internet and even publish fancy live charts and dashboards. All for free!

Benefits include:

      • You can view the data in real-time from any computer on the internet.
      • You can make and publish live charts and graphs.
      • You can easily share the data, or keep the data private.
      • You can analyze the data directly using spreadsheet functions and Apps Scripts.
      • You can set up triggers to email/text you based on incoming data.

  • You can log data from any platform that can generate an HTTP GET or PUT request (pretty much any thing that can access web pages).
  • You do not need any servers or port forwarding.
  • The logging device does not need to know your Google username or password – just a special URL that logs directly to a specific sheet.
  • Lots of storage space.
  • Free!

For an example, open this logging example sheet in a different browser window and scroll down to the bottom. Next, go here….

You should see a new line of data appear at the bottom of the spreadsheet.  Next try this URL…

If your logging client is running Linux, you could execute a command like…

curl -k "$(date +%F@%T)&TEMP=68.2&WIDTH=190&SPEED=55"

(note that the quotes around the URL are important)
Get the idea? You can play more here. Note that knowing the URL does not let you modify the spreadsheet- only append data to it. Conversely, you also do not need to have access to the logging spreadsheet to be able to add data to it using the URL – you do not even need to be logged into Google (good for embeded devices).

Setting up your own Google spreadsheet for logging

Thanks to several bugs and missing features, this turns out to be a pain to set up. Luckily, you can forget all the hassle once it is set up – and I am about to tell you exactly how to set it up.

You will need a free Google Drive account to set up your own logging spreadsheets.


    1. Open this template spreadsheet…

  1. Choose “Make a copy” to make a copy into your account…image
  2. From the new copy you just make, choose “Setup Logging –> Setup Script”…image
  3. Follow the prompts to authorize the script.
  4. Choose “Tools->Script Editor”…image
  5. Choose “Deploy as Web App”…
  6. Click “Save New Version”…image
  7. Select “Anyone, even anonymous” for “Who has access to the app”…image
  8. Click “Deploy”…
  9. You win! Copy the resulting URL. Make sure you get the whole thing.image

You can now use this URL to log data your spreadsheet!

How it works

You can add any parameters you want to the end of the URL you got above. The script will automatically check the top row of the spreadsheet for parameter names and add any it does not find. It will then add a new row to the bottom of the spreadsheet with the parameters in the appropriate columns. The order of the parameters in the URL is not important.

If you are using HTTP GET method (as shown above) to make the call, you might want to add a timestamp or sequence number field to avoid getting dupes if the GET request gets resent.

Alternately, you can also do an HTTP POST to the same URL and the data inside the POST will be appended to the spreadsheet. You can even use the logging URL as the action for HTML forms, so you can use this technique to make your own more flexible version of Google Forms.


Q: Couldn’t you just use or something like that?
A: I bet you could, but spent a good hour trying to get set up and failed. I tried a few others too, and nothing is as convenient, easy, or free as a Google spreadsheet – and once you have the data in the spreadsheet there is a ton of other stuff you can then do on Google (sharing, triggers, analysis, graphing) very easily as well.

Q: How about TEMBOO?
A: Temboo is cool and in theory could make all this stuff very simple, unfortunately it has two deal breakers for me.

First breaker is that they require me to put my Google username and password into the program that will go into the embedded device. Yikes! I don’t tell my google password to anyone – much less put it in plaintext inside a portable device. They could have mitigated this problem by doing OAUTH on their servers to get access to the spreadsheet, and then given you a token to put into the device that can only do the specific things you specify and could easily be monitored and revoked.

Second breaker is their pricing model. The first 1,000 calls/month are free and then it starts costing money for every call. This just doesn’t work for an application where I want to log a data point every few seconds- especially when you can do it faster, more reliably, and more securely yourself. It especially doesn’t work when I want to have LOTS of logging things in my life.

Q: Can’t you just script the above steps for me in Apps Script and give me an easy menu choice?
A: Google Apps Script does have a a Service Class that do exactly that, unfortunately is has been broken for more than a year and there is no sign it will be fixed anytime soon.

Q: How can I control who can log to my spreadsheet?
A: First off, you can control who you give the logging URL to. It is presumed that google generates these URL securely, so as long as you do not publish the URL then no one can push data into your spreadsheet.

But what if you wanted to make a logging application where lots of independent devices could all add data to a single spreadsheet? In that case, you could a token parameter to the URL and a bit of code in the processing script to check the incoming token against a private list of authorized tokens. You’d only add the data if the token was found on the list. You’d then add a new token to the list (possibly automatically via another script) each time you wanted to authorize a device, and you could always delete the token (and any data it had previously logged) if you suspect it was compromised.

UPDATE 7/14/14 – Looks like SparkFun has just set up a service that also lets you log sensor using a simple URL-based scheme. It looks well done, but I think I still prefer using a Google spreadsheet because you get way, way more space and you can easily do stuff with the data like calculations, graphs, and alerts.

NeoPixels Revealed: Why you should give your bits room to breathe

This is Part 4 in a series. Read Part 1 here.

On the perils of trying to pack your bits too tightly

While superficially, it might seem like a good idea to try to pack your bits in as tightly as possible. You want to update your string as rapidly as possible to reduce transmission time. Unfortunately, this can actually cause some hairy problems that get worse as your strings get longer.

Remember how signal reshaping will take a short bit and stretch it out to make it into an “ideal” bit? If you try to be cute and pack your bits in as tightly as you can, eventually these stretched bits are going to use up the gap before the next bit and violate the VDL minimum. When this happens, you see a glitch on the string.

You might think that as long as you keep your bits at least as long as the “ideal” size, then they will not get stretched and you will avoid glitches.  Unfortunately, the “ideal” bit size varies from pixel and even changes over time for a single pixel depending on temperature and voltage

Heck, you can even have cases where a chip will turn tightly packed 1-bits into 0-bits thanks to signal shaping. Take at look at this capture…

Valid 1-bit getting subverted into 0-bit by signal shaping

Valid 1-bit getting subverted into 0-bit by signal shaping

Here we were trying to to squeeze our 1-bits to be as small as possible while still meeting the letter of the law. The bottom blue trace shows a perfectly valid 560ns wide 1-bit (spec says 700ns ±150ns = 550ns min) coming in, but the purple trace shows it going out the other side now reshaped to  a 376ns wide 0-bit. Why did it do that? Maybe it was hot. Maybe it was just having a bad day. It doesn’t really matter because if this happened to you, what you would see is that all of the LEDs past this one went black – a symptom that would surely have you wasting lots of time looking for a power or signal break.  And the longer your strings, the wider the range of behavior like this you are likely to see.

Give your 1-bits and your gaps a little room the stretch their legs and it will save you all kinds of intermittent and hard to debug problems

The moral of the story: Give your 1-bits and your gaps a little room to stretch their legs and it will save you all kinds of intermittent and hard to debug problems. You don’t want to stretch your 0-bits, however, because then they might become 1-bits (see T0H maximum), instead you can stretch the gap after the 0-bit to give it some room to expand into. For some simple bit-pampering code, check out…

NeoPixels Revealed: How to (not need to) generate precisely timed signals