TCD1304 USB Spectrograph Board
An Arduino can't keep up with a CCD using the internal analog to digital converter, but with an external 8-bit half-flash ADC it's no problem at all. The ADC is the ADC0820, which can digitize up to 666k samples per second. The sample rate here works out to be just over 222kSPS using the code below. The 3694 pixel frame takes approximately 250mS to read, and download via USB.
Are you looking for the Legacy ADC0820 Spectrograph? It uses a serial interface, which takes 16 times longer to download, has a power supply onboard and takes a 9Vdc wall wart. This new version uses an Arduino Pro Micro as a parallel-to-USB converter, giving 1/4 second downloads, and is powered by the USB bus. Overall cost is nearly identical.
Or are you looking for an Arduino Uno version, which reads the CCD in 800 points, rather than the 3694 pixels read by the two above? If you are not as concerned with spacial resolution, this might be the one.
Hardware
Microcontrollers
The Arduino Uno doesn't have enough RAM for a 3648 pixel buffer. The ATmega1284 has 16kB RAM. The ATmega1284 has two responsibilities - get the data out of the CCD and into a RAM buffer, and transfer the data to the Pro Micro on demand.
The Arduino Pro Micro reads the buffered data via a parallel interface, and sends it in 64-byte packets to the host. The 64-byte buffer was chosen because it is more efficient for the COBS packetizing code and can evenly divide into 3648, the number of active pixels in the TCD1304. The Arduino Pro Micro has a full speed (12Mbps) interface to the host computer, providing very short download times.
Analog to Digital Converter
The project uses the ADC0820CCN, which is in a DIP package. The ADC0820 ADC has + and - reference inputs designed to set the range of the ADC, so you can focus on the interesting portion of the input range, keeping full 8-bit resolution. The ADC0820BCN part has ±1/2 LSB linearity. The ADC0820CCN has ±1 LSB linearity. Either is fine for this project. The ADC0820BCN costs considerably more than the ADC0820CCN.
Linear Array CCD
The linear CCD is a Toshiba TCD1304 3648 pixel sensor which requires a supply voltage from 3V to 5.5V. The sensor is driven directly by the ATmega1284 microcontroller, and the analog output is buffered by a PNP transistor. The TCD1304AP CCD is available on ebay or aliexpress for anywhere from $8 to $40. Mouser has new ones (TCD1304DG) for $36.
PC Board
The PC boards came from PCBWay.com, and they turned out perfect. I use them because they have perfected making prototype and limited production pc boards. The largest board I have had them make was 100mm x 150mm, and the smallest was 32mm square. And many in between. Every time I get the same high quality and fast turnaround. I recommend them to anyone who needs a fast circuit board and is not willing to sacrifice quality to get it.
The majority of the rest of the parts came from Mouser.com, except the ADC0820BCN, which came from TEDSS.com. The ADC0820CCN is available at many outlets, including ebay.
Schematic Diagram
Description
Timer2 on the ATmega1284 generates an 888.88kHz square wave to be used as the MCLK signal. The code that
toggles the clocks on the CCD and ADC does so in cadence with the MCLK signal. The clock edges are aligned
by an adjustment of the timer one cycle before the line is read. MCLK runs continuously. The SH,
ICG, WR, and RD signals are only asserted when the CCD is being read.
The output of the CCD is buffered by a PNP transistor before being sent to the ADC. The ADC range is configurable, maximizing the digital resolution. Since the signal is inverted (lower is brighter) it requires the code to subtract the pixel value from 255 to put it right.
ICSP Connector and JP1
At the pin 1 end of the ATmega1284 microcontroller is a programming connector. Beside pin 3 is a jumper marked JP1. The
programming connector is for burning the code into the ATmega1284. Jumper JP1, when installed, passes
programmer power to the circuit to allow programming with no other power source. When the JP1 jumper is removed,
the programmer is isolated. That is how it should normally be to keep from having collisions between the USB's 5V and
the programmer's 5V.
RV1 and RV2
Trimmer pots RV1 and RV2 are for adjusting the black level and white level respectively. Don't use cheap pots.
Use Bourns if you can get them. The cheap ones act really flaky at the ends of their ranges, and can give you fits trying to fine tune
the calibration.
RST is the reset on the Pro Micro and the ATmega1284. Just in case you are working on code, and it hangs.
It's better than disconnecting the USB cable and reconnecting.
The only problem with the PC boards is with my design. I discovered the need to connect the ATmega1284 reset and the Pro Micro reset after I built the first one. So the prototype has a kludge on the bottom. But that's what prototypes are for. I'll get new boards when I'm done butchering this one. Another issue is my soldering iron does not have quite enough power to solder the component leads that connect to ground planes. My 40-year-old Weller bit the dust, my new rework station has a weak iron, and the thermal pads don't have enough gap. All will be fixed by the time the new boards are done.
Assembly
Assembly is straightforward, but be aware that the CCD requires a 22 pin socket with a width of 400 mils (10.16mm). You can use female machined pin headers, which I have done in a pinch, but this time I did use a socket.
If you are wiring it up on a proto board, Be careful to keep the digital clocks and analog wiring away from each other where possible. Use 0.1µF bypass capacitors at every Vcc connection. Keep all the digital wires as short as practical to keep them from radiating more than necessary. Be especially careful when routing the MCLK signal near analog areas. That is the only external signal that is toggling while the ADC is converting the analog signal.
The Pro Micro will need to have its power LED removed or disabled, or just covered well. It puts out enough light to pollute any measurements.
Software
Use the MightyCore boards manager plugin by MCUdude for the ATmega1284. The board will be ATmega1284 and the variant will be the 1284 or the 1284P, depending on which one you have. Pinout is "standard" and the frequency is 16MHz. You can add the MightyCore boards to the boards manager by adding the following url to the "Additional Boards Manager URLs" textbox in Arduino preferences:
https://mcudude.github.io/MightyCore/package_MCUdude_MightyCore_index.json
Then just open the Boards Manager under Tools->Board->Boards Manager and go to the bottom to install the MightyCore boards. The selection of boards will increase considerably to include all the ATmegaxx4 chips, ATmega16/32, and ATmega8535. I didn't use a bootloader because the serial port is busy talking with the Pro Micro, so you will need an AVR programmer to burn the code into the ATmega1284. Hold the shift key while clicking the program button. They're a few dollars on ebay. The board uses a 10-pin ICSP connector.
The source code for the linear CCD application is very carefully crafted in places to maintain the phase relationship between all the CCD clocks and stay true to the specification. It takes a logic analyzer to validate the results of any code changes in the "readLine()" function.
The code to transfer data to the Pro Micro for shipment to the host via USB gets programmed into the Pro Micro via USB. The Pro Micro presents itself as a Leonardo. The port, at least on a linux machine, is /dev/ttyACMx, where x is a number 0, 1, ..., etc. It gets programmed normally (no shift key).
#include <util/delay_basic.h>
// ADC RD signal pin 17
#define RD 0x08
// ADC write signal pin 18
#define WR 0x10
// CCD Shift Gate pin 19
#define SH 0x20
// CCD Integration Clear Gate pin 20
#define ICG 0x40
// CCD Master clock pin 21
#define MCLK 0x80
// CCD and ADC clocks
#define CLOCKS PORTD
#define CLOCKP PIND
// ADC data
#define ADATA PINC
#define HDDATA PORTA
#define HDDDR DDRA
#define HCDATA PORTB
#define HCPINS PINB
#define HCDDR DDRB
#define HCMSK 0x01
#define DA 0x01
#define DT 0x02
#define BUSY 0x04
#define PIXEL_COUNT 3694
uint8_t pixBuf[PIXEL_COUNT];
char cmdBuffer[16];
int cmdIndex;
int cmdRecvd = 0;
int exposureTime = 10;
bool looping = false;
//***********************************************************************
// setup() Init the clock I/O, clock freq, serial port.
//***********************************************************************
void setup() {
// Initialize the clocks.
DDRD |= (WR | SH | ICG | MCLK | RD); // Set the clock lines to outputs
CLOCKS = (ICG + RD + WR); // Set the ICG, RD, and WR high.
// Setup the ADC data port as inputs.
// DDRC = 0;
// Enable the serial port.
Serial.begin(19200);
// Setup timer2 to generate an 888.88kHz frequency on pin 21
TCCR2A = (0 << COM2A1) | (1 << COM2A0) | (1 << WGM21) | (0 << WGM20);
TCCR2B = (0 << WGM22) | (1 << CS20);
OCR2A = 8;
TCNT2 = 1;
// Initialize the parallel handshake.
HDDDR = 0xff; // All outputs.
HCDDR = HCMSK; // 3 handshake lines - one output, two input.
HCDATA &= ~DA; // No data available.
HDDATA = 0; // Clear the data.
for (int x = 0; x < sizeof(pixBuf); ++x) {
pixBuf[x] = x & 0xff;
}
}
//***********************************************************************
// readLine() Reads the line of pixels in the TCD1304 and digitizes it.
//***********************************************************************
void readLine() {
// Get a pointer to the buffer.
uint8_t *buf = pixBuf;
int x = 0;
uint8_t scratch = 0;
// Disable interrupts or the timer will get us.
cli();
// Synchronize with MCLK and
// set ICG low and SH high.
scratch = CLOCKS;
scratch &= ~ICG;
scratch |= SH;
while(!(CLOCKP & MCLK));
while((CLOCKP & MCLK));
TCNT2 = 0;
_delay_loop_1(1);
__asm__("nop");
__asm__("nop");
__asm__("nop");
__asm__("nop");
__asm__("nop");
CLOCKS = scratch;
// Wait the remainder of 4.5uS @ 16MHz.
_delay_loop_1(22);
__asm__("nop");
__asm__("nop");
// Set SH low.
CLOCKS ^= SH;
// Wait the remainder of 4.5uS.
_delay_loop_1(23);
// Start the readout loop at the first pixel.
CLOCKS |= (RD + WR + ICG + SH);
do {
// Wait a minimum of 250nS for acquisition.
_delay_loop_1(2);
// Start the conversion.
CLOCKS &= ~WR;
_delay_loop_1(3);
CLOCKS |= WR;
// Wait the rest of 2uS for conversion.
_delay_loop_1(9);
// Read the result.
CLOCKS &= ~RD;
_delay_loop_1(3);
__asm__("nop");
__asm__("nop");
*buf++ = ADATA;
// Set the clocks back to idle state
CLOCKS |= RD;
// Toggle SH for the next pixel.
CLOCKS ^= SH;
} while (++x < PIXEL_COUNT);
CLOCKS = (ICG + RD + WR);
sei();
}
//***********************************************************************
// sendData() Sends the video data from the pixel buffer in RAM to the
// Pro Micro over a parallel interface. The Pro Micro reads
// the data as 57 packets of 64 bytes each. We send it as a
// continuous stream of 3648 bytes.
//***********************************************************************
void sendData(void) {
int x;
for (x = 30; x < 3678; ++x) {
while (HCPINS & BUSY);
HDDATA = 255 - pixBuf[x];
HCDATA |= DA;
while (!(HCPINS & DT));
HCDATA &= ~DA;
}
}
//***********************************************************************
// loop() Main program loop. Looks for commands on the serial port,
// then executes them. The Pro Micro send commands over the
// serial port, and the data is returned through a parallel
// port.
//***********************************************************************
void loop() {
int x;
char ch;
if (cmdRecvd) {
if (cmdBuffer[0] == 'r') {
// Clear any residual charge.
for (x = 0; x < 3; ++x) {
readLine();
}
// Expose.
delay(exposureTime);
// Read and send data.
readLine();
sendData();
} else if (cmdBuffer[0] == 'e') {
exposureTime = atoi(cmdBuffer + 1);
if (!exposureTime) {
exposureTime = 1;
}
}
memset(cmdBuffer, 0, sizeof(cmdBuffer));
cmdRecvd = 0;
cmdIndex = 0;
}
while (Serial.available() > 0) {
ch = Serial.read();
if (ch != 0x0a) {
cmdBuffer[cmdIndex++] = ch;
} else {
cmdRecvd = 1;
}
}
}
#include <SoftwareSerial.h>
#include <PacketSerial.h>
#include <util/delay_basic.h>
// Digital interface to CCD controller 1284
#define low_nibble_pins PIND
#define low_nibble_ddr DDRD
#define hi_nibble_pins PINF
#define hi_nibble_ddr DDRF
#define hndshk_port PORTB
#define hndshk_pins PINB
#define hndshk_ddr DDRB
#define data_available 0b00000010 // Input from 1284 (B1)
#define data_taken 0b00000100 // Output to 1284 (B2)
#define busy 0b00001000 // Output to 1284 (B3)
#define v_buf_size 64 // 64 pixels per read
#define v_buf_pages 57 // 57 64-byte buffers per line
int RXLED = 17;
byte v_buf[v_buf_size] = {0};
int v_buf_cnt = 0;
int exp_ms = 10;
// Intercept 'Serial' to translate packets to and from COBS.
PacketSerial_<COBS, 0, 256> host;
// The serial interface to the ATmega1284 for commands.
SoftwareSerial device(9, 10);
//***********************************************************************
// reset_v_hndshk() Sets up the handshake lines for the parallel
// interface.
//***********************************************************************
void reset_v_hndshk() {
hndshk_ddr = data_taken + busy;
hndshk_port &= ~(data_taken + busy);
hndshk_port |= busy;
}
//***********************************************************************
// reset_v_port() Sets up the parallel port of the interface to
// the ATmega1284.
//***********************************************************************
void reset_v_port() {
low_nibble_ddr = low_nibble_ddr & 0xf0;
hi_nibble_ddr = hi_nibble_ddr & 0x0f;
}
//***********************************************************************
// read_pixels() Reads one 64-byte buffer of pixels from the
// parallel port and buffers it.
//***********************************************************************
byte read_pixels() {
byte timeleft;
int x;
for (x = 0; x < v_buf_size; ++x) {
hndshk_port &= ~busy;
timeleft = 255;
while (!(hndshk_pins & data_available)) {
_delay_loop_1(150);
if (!--timeleft) {
break;
}
}
_delay_loop_1(2);
if (timeleft) {
v_buf[x] = (hi_nibble_pins & 0xf0) + (low_nibble_pins & 0x0f);
hndshk_port |= (busy + data_taken);
} else {
hndshk_port |= (busy + data_taken);
break;
}
timeleft = 255;
while ((hndshk_pins & data_available)) {
_delay_loop_1(50);
if (!--timeleft) {
break;
}
}
hndshk_port &= ~data_taken;
}
hndshk_port &= ~data_taken;
return(x == 64);
}
//***********************************************************************
// read_line() Reads the entire line of 57 64-byte packets from
// the parallel port and sends them one at a time to
// the host PC.
//***********************************************************************
bool read_line() {
bool complete = true;
for (int page = 0; page < v_buf_pages; ++page) {
if (read_pixels()) {
host.send(v_buf, sizeof(v_buf));
hndshk_port &= ~busy;
} else {
complete = false;
break;
}
}
return(complete);
}
//***********************************************************************
// onPacketReceived() Called by the COBS code via 'host.update()'. It
// passes the received command along, then in the
// case of the read command, it goes to wait for the
// data to be sent from the ATmega1284.
//***********************************************************************
void onPacketReceived(const uint8_t* buffer, size_t size)
{
byte buf[32] = {0};
int bytes_written = 0;
if (buffer[0] == 'r') {
bytes_written = device.print((char *)buffer);
bytes_written += device.print("\n");
delay(exp_ms + 64);
read_line();
} else if (buffer[0] == 'e') {
exp_ms = atoi(buffer+1);
bytes_written = device.print((char *)buffer);
bytes_written += device.print("\n");
} else if (buffer[0] == 'd') {
bytes_written = device.print((char *)buffer);
bytes_written += device.print("\n");
}
}
//***********************************************************************
// setup() Inits the parallel port, and the serial ports. The 'host'
// serial port is set to 9600 baud, which is ignored since
// the data goes at USB Full Speed (12Mbps). The software
// serial port runs at 19200 for the command interface to
// the ATmega1284.
//***********************************************************************
void setup()
{
reset_v_hndshk();
reset_v_port();
// The serial connection to the host via USB.
// Baud rate is ignored.
host.begin(9600);
host.setPacketHandler(&onPacketReceived);
// The serial connection to the CCD controller 1284.
pinMode(10, OUTPUT);
device.begin(19200);
}
//***********************************************************************
// loop() Just checks the serial port for updates from the host,
// and sends the data back.
//***********************************************************************
void loop()
{
host.update();
}
Timing
At time 0s:0ms:0µs you can see the MCLK is stretched a little. That is to get the ICG, SH, WR and RD lines in sync with the timer, which generates the 888kHz MCLK signal. The code has to maintain this timing relationship throughout the entire read process, reading one pixel every four MCLK cycles. The readLine() function is riddled with assembly language "NOP" code to keep the timing accurate. Keep in mind that adding even a single machine instruction requires re-factoring the read timing.
The interface between the ATmega1284 and the Arduino Pro Micro is 8-bit parallel, with handshake consisting of DATA_AVAILABLE,
DATA_TAKEN, and BUSY:
When the 1284 has received the read command, it clears the CCD, delays for the exposure time, and reads the CCD into its buffer. This
all takes (16mS x 4) + the exposure time. Then it starts looking for BUSY to be low. When it is, the 1284 puts a byte of
data on the data port and asserts DATA_AVAILABLE. When the Pro Micro sees DATA_AVAILABLE, it reads the data,
then asserts DATA_TAKEN and BUSY. On seeing DATA_TAKEN, the 1284 drops DATA_AVAILABLE,
and waits for a low BUSY to start the next transfer. The Pro Micro drops DATA_TAKEN, and after processing,
drops BUSY. This pattern is repeated 64 times to fill one USB buffer. During the subsequent USB transfer, BUSY
remains high. The 64-byte buffer is transferred 57 times to do a complete line of pixels. The whole process takes around 231mS for a 1mS
exposure.
The Sampler Program
The Sampler program is written for Python 3. You must also install pyserial pip install pyserial, cobs pip install cobs and tkinter. Do a search on how
to install tkinter on your operating system. The first couple of results are good.
A few lines up from the end of the python code is a line:
app = Application(master=root, port="/dev/ttyACM0", exposure=50)
It should match your serial port. If you are using Windows, change that to something like:
app = Application(master=root, port="COM3", exposure=50)
Again, make it match the serial port you are using. If the app ends with return code 1, it didn't find the port. When you click Sample, it draws a graph of the CCD output, but also makes a file in the program directory named 'ccd.csv'. That file can be imported into a spreadsheet application for further processing, including making a graph at the full resolution of the CCD.
The horizontal resolution of the sensor, and the data returned, is three times the resolution of the graph. The csv file is written at full resolution. The pixels are binned into 3-pixel wide buckets for the graph. The graph is marked in nanometers, from 400 to 700. It is not a linear scale, and it is dependent on the orientation of the different parts of the spectrograph. See the bottom of the TCD1304 Arduino CCD Transmission Spectrograph page for details.
The buttons are:
- Sample - Read the CCD and put the trace on the screen.
- Clear - Clear all traces from the screen.
- Quit - End the program.
- Exposure - An integer value from 1 to 1000. The milliseconds to expose.
When you click on Sample, it draws the graph. It does not erase the previous line. The Clear button does that. With the USB interface, the time from clicking the button to drawing the graph can be as little as the exposure time + 230mS.
If you ever get what looks like a saturated signal (flat on top), first just redo the exposure. CCDs gather light whether you read them or not, so leaving it alone for 30 minutes is like taking a 30-minute exposure. The code reads the frame 3 times before exposing, but sometimes that isn't enough to get all the electrons out of the system. If that is the case, keep sampling, and each trace will be a little more normal than the previous one.
Calibration
To calibrate, start by setting the RV1 pot, "Vref (+)", all of the way clockwise. Set RV2, "Vref (-)" all of the
way counter-clockwise. This sets the black level at 5V and the white level at 0V.
Get a piece of black electrical tape long enough to cover the ccd + 1/4 inch on each end. Cut a slit across the tape, but not
all the way - just in the center. Use an X-acto knife or similar. Then cover the CCD with the electrical tape in a room
with dimmed lighting. The tape will come off without damaging the CCD. You should see a spike when you make a sample. If not,
increase the exposure time. If the spike looks more like a square pulse, decrease the exposure time. If you rub your finger
down the length of the tape on the CCD, starting at the slit, it will open the slit wider, if you need it. Rubbing your finger
toward the slit will tend to close it.
Adjust the exposure time until you get a maximum linear output from the spike. When the spike stops linearly increasing in height, stop increasing the exposure time. In the following image, you can see the curves getting linearly higher in amplitude, then going non-linear on the last step. I started at 100mS and worked up by 50mS per step to 250mS. 200mS was the last linear improvement, so I chose that as the working expoure time for the calibration procedure. The non-linearity is due to sensor saturation.
Clear and take a single sample at the current settings.
Adjust "Vref (+)" counter-clockwise, taking samples along the way, until the bottom line of the signal just reaches about to the bottom of the graph. It may take several turns of the pot to get there. If you overshoot, turn the pot the other direction to bring it back up. You want to see all the noise, so don't try to get a flat, straight line on the bottom.
Clear and take a single sample at the current settings.
Turn "Vref (-)" clockwise, taking samples along the way, until the tip of the peak just touches the top line. If you overshoot, you can bring it back down. It will bring the bottom up a bit, but we will tune that out later.
Clear and take a single sample at the current settings.
Go back to the "Vref (+)" pot and turn it counter-clockwise to bring the bottom back down.
Clear and take a single sample at the current settings.
Finally, if needed, back to the "Vref (-)" pot to pull the top up again. I didn't need it in this case, but only because I lucked out and hit it right the first time. If you want to get closer, keep repeating these last two steps until you are satisfied. The amplitude is not calibrated, but it is linear, if the last linear exposure time was chosen above. What this calibration does is increase the digital resolution to use as many of the 256 steps as possible.
You must adjust the pots in that order because, as you saw above, they interact with each other. That is by design. The maximum voltage you can put on Vref(-) is the voltage on Vref(+), and the minimum is ground, so changing Vref(+) changes the range and the current value of Vref(-).
When you remove the tape, clean the CCD glass with alcohol to get any residue off of it.
From now on, you use the exposure time to increase or decrease the range of the signal. If the line goes off the top of the graph, lower the exposure time, and if it falls short, increase the exposure time. If 1000mS isn't long enough, you can change it in the Python Sampler and in the Arduino IDE (both) to be whatever you need it to be. Noise increases with exposure time. If you need it to be shorter, we may need to experiment with microsecond timers. Send me some info on what you are doing via the Contact Form and I'll see what can be done.
Mechanics
Here is where we all go our separate ways. If you want a Raman spectrometer, you will be adding a laser, a grating, and a few mirrors. If you want a barcode reader, you will add a lens. My particular use for this was to characterize dichroic color separation filters used in CCD astrophotography. To do that, I needed a collimated light source, a stage for the filter, a slit, a transmission grating, and the board. I used a 1000 lines per mm Edmund Optics holographic grating in a 2" x 2" slide card.