TXT-Controller controls my Nikon SE microscope.

[fotoopa]

I've been working on my microscope project for several months. The project is in its final phase. The aim is to control a converted Nikon microscope unit via RoboPro. A Nikon D7100 camera with a tube lens 70-200mm is mounted on the microscope. On the microscope itself, different lenses can be mounted according to the desired magnification. Currently I own a 4X, a 40x and a 100x microscope lens. The 40x and the 100x are oil lenses for large magnifications. Resolution of the images are in the um area. They are used for viewing of preparations.
The 4x microscope lens is more suitable for small insects. A full image has about 11mm image view. The depth of sharpness of the images with such a lens is around 20 um. To photograph an insect part from 1mm to 2mm height you have to shoot at least 50 to 100 images. Once you have recorded all these images, they will be compiled into one sharp image using stacking software. This is done with zerene stacker software.

I use the TXT-Controller to control the stepper motor, flash and camera via the I2C. The flash contains 4 COB leds each 125W peak. The higher the power, the shorter the exposure time you can take. This is a great advantage to avoid blur due to small vibrations of the system. Even the camera shutter movement gives already strong vibrations. This is why certain delays have been used in the timing of the camera.

The TXT-Controller has the great advantage of a perfect interface with your user screen. You can put a lot of controls, values, and functions on your screen. Programming goes very smoothly and flexible. From the TXT-Controller I only use the I2C. All other functions are performed in real time by an FPGA which is programmed as a large intelligent I2C chip. Timing is done in usec. The I2C FPGA can deliver all data back to the TXT-Controller. Position of the stepper motor, the settings of the flash times, the number of pictures to be taken, the step per picture, etc. The microscope is connected to the I2C chip via a fast SPI line. The refrech time is 10 usec. All functions are executed in the I2C chip, so the TXT-Controller itself does not need a critical function anymore.

The controller housing and flash are compatible with Fischertechnik elements via 3D printing. On my Flickr web all schematics are available. I would have to give an overview drawing but this takes a lot of time. I'll make this later.

Some images of the modified microscope:

HD: https://www.flickr.com/photos/fotoopa_hs/48365354531

HD: https://www.flickr.com/photos/fotoopa_hs/48365489737

HD: https://www.flickr.com/photos/fotoopa_hs/48365489612

The PC screen of the TXT-Controller interface is shown here:

HD: https://www.flickr.com/photos/fotoopa_hs/48329195426

The result of a shoot after stacking:

Hd: https://www.flickr.com/photos/fotoopa_hs/48329254706
Note that the latter image is actually a full 3D HD image. How to handle it is shown in the picture on my Flick web.
I can give more details if desired. The problem is that this requires a lot of preparation.

Frans.

[Burkhard]

Wow, that looks amazing and such a nice practical use of fischertechnik interfacing to "real world" things doing something useful.

A couple of questions:
1. I understand that you need to stack the images because with the magnification you are employing, the depth-of-field is very short and thus only parts of the insect at a narrow range of distance to the lens are in focus. So you have a whole bunch of identically-framed images wherein each has a different portion of the image in focus. Now here's my question... How do you "tell" the software stacker which parts of each image to use and which parts to throw away? Or does it figure that out itself somehow?

2. You mentioned at the end that the final image is "3D". By this, do you mean stereoscopic, such there is a left image and a right image, differing in perspective slightly (≈ the human interocular distance). If you do have L / R image pairs, I'd like to see some. I've shot some stereo images myself and use Stereo Photo Maker software to view them.

interesting stuff!
burkhard

[fotoopa]

Wow, that looks amazing and such a nice practical use of fischertechnik interfacing to "real world" things doing something useful.
A couple of questions:
1. Now here's my question... How do you "tell" the software stacker which parts of each image to use and which parts to throw away? Or does it figure that out itself somehow?

Yes, that is the task of Zerene stacking software. This is done automatically in a batch file. There are several parameters you can set. You can also ask him to create a 3D image. Zerene requires a good CPU preferably with many cores. I use Windows 7 64 bit with an I7 intel cpu and 16 Gb ram memory. He uses all the cores if you let him. I further process the 3D image with stereo Photo Maker.

2. You mentioned at the end that the final image is "3D". By this, do you mean stereoscopic, such there is a left image and a right image, differing in perspective slightly (≈ the human interocular distance). If you do have L / R image pairs, I'd like to see some. I've shot some stereo images myself and use Stereo Photo Maker software to view them.

The full HD image on Flickr is in fact a full 3D HD image. But Flickr does not support a .mpo file. Because a .mpo file consists of 2 consecutive .jpg images, you can still upload the .mpo file. However, Flickr will convert the extension to .jpg and will only show the first image. If you download my original image from Flickr and you change the extension from .jpg to .mpo you will get the original 3D mpo file suitable for real 3D TV or monitors.

Another important point is the ability to see live view images on your monitor to adjust the start and end point for the recordings. The Nikon D7100 supports this live view together with the Control My Nikon software. On your large screen, you can see the image live and where the pictures will be sharp. Once the start and end points have been set, you know how many images need to be taken. The step per image for a lens is a fixed fact. I usually use a step of 20um per image. With the RoboPro software interface I can quickly navigate to a certain point and take a test picture to set the exposure. In the self-built flash module are 4 COB power LEDs each 125 W peak. By setting the flash time per COB led you can give a direction to the light. This allows you to set the best exposure with the test shot.

Then it's time for to record the series of images. Normally this is around 100 images. The RoboPro program does the recording automatically via the Run mode. The Nikon camera takes the pictures in mirrorup mode. This causes the mirror to be flipped over first and only 100 msec later the shutter is opened. As a result, many vibrations have already disappeared. Once the shutter is open, another 95 msec is waited to control the flashes. Partly because of this extra waiting time and the very short flash times, the recorded image is vibration-free. Most flash times are between 100 and 500 usec. The cycle time for the images is about 3 sec. Most of the time is needed to send the Raw data (.nef file) to the PC. All images will be sent directly to the disk of the computer, ready for the next batch processings. Once all pictures have been taken, the data from .nef should be converted to .tif. This is done with Capture NX 2 also with a batch file command. The result is 100 .tif images of 8 or 16bit resolution at 4000x6000 pixels.
The .tif files are then sent to the Zerene program, which will compose the image into a full picture with 3D capability. That is a very large amount of data to be processed, several GB. Finally, come Stereo Photo maker to optimize the 3D image and put it in the right format.

The end result is a full HD image in 3D. I use an Asus 27 inch 120Hz 3D monitor with Invidia glasses for these images.

Update:
In 2014 I already used the camera timing for my old setup. The new timing in the microscope is made on this basis.

HD: https://www.flickr.com/photos/fotoopa_hs/13670954895

A short video how the focus change by moving the lens via RoboPro software and the TXT-Controller.
https://youtu.be/2URJ5Tncp9A

Frans.

[Burkhard]

2. You mentioned at the end that the final image is "3D". By this, do you mean stereoscopic, such there is a left image and a right image, differing in perspective slightly (≈ the human interocular distance). If you do have L / R image pairs, I'd like to see some. I've shot some stereo images myself and use Stereo Photo Maker software to view them.

The full HD image on Flickr is in fact a full 3D HD image. But Flickr does not support a .mpo file. Because a .mpo file consists of 2 consecutive .jpg images, you can still upload the .mpo file. However, Flickr will convert the extension to .jpg and will only show the first image. If you download my original image from Flickr and you change the extension from .jpg to .mpo you will get the original 3D mpo file suitable for real 3D TV or monitors.
Frans.

Ahhh, yes, it seems you explained the *.jpg→*.mpo trick on the Flickr page, but it was further down and I hadn't scrolled far enough to see it. Sorry about that!

I downloaded the image and examined it on my laptop through both cross-eyed viewing and as an anaglyph—looks amazing!
My father (long a stereophotography fan) has both a 3D Toshiba laptop and a 3D Samsung TV, each with Bluetooth shutter glasses. When I see him next, I'll try to get your fly image on one of his devices.

One final (I think) thing I don't understand. Normally, to take a 3D image, there are actually 2 separate exposures, one from a left lens and one from a right lens which differ slightly. You have the two different images at the end, but how is that achieved? Isn't the whole stack of exposures taken at the same point, just with different portions in focus? Don't you need to translate the camera (or the subject) in the x-direction a smidge to achieve a different view angle for each eye?

This is really nice work, Frans. Thank you for sharing!

P.S. [later edit] I see now the *many* amazing images you have on your Flickr page. I will be sure to take a look at these with my Dad, who has been a stereophotography fan (although nothing like you are!) since the 1950s with a Stereo Realist.

[fotoopa]

Glad to hear that you succeeded in seeing the real 3D images. There are not many users who have the opportunity to watch this on the right monitor or TV. If you don't use cross-view, the 3D glasses are always a bit awkward. Still, it really pays to use good 3D images with the right tools. It gives you an extra dimension. You'll discover so many small details when using 3D images.
Yes, there are a lot of 3D images on my Flickr page. I have been doing 3D high speed, macro and micro photography for years. A lot of technical information is also online, hardware, software, setups, tool etc. I'm sure your father will find something interesting...

One final (I think) thing I don't understand. Normally, to take a 3D image, there are actually 2 separate exposures, one from a left lens and one from a right lens which differ slightly. You have the two different images at the end, but how is that achieved? Isn't the whole stack of exposures taken at the same point, just with different portions in focus? Don't you need to translate the camera (or the subject) in the x-direction a smidge to achieve a different view angle for each eye?

That's exactly what the Zerene stacking software is doing. Because you take multiple images in height, the software knows which part of the image should be higher (closer to your viewing point). In the 3D settings you can now set a shift in % both in X or Y direction. This shift determines your 3D effect. Normally I use -2 or -3% in X direction which corresponds to a ratio of 1/30. Entering a to high value does not always result in good natural images. You only need to run the recording sequence once. Afterwards, both 3D images are generated with Zerene. You must be able to take a minimum number of images otherwise the software will not work properly. As soon as you have 60 or more images during the recording, you can generate nice 3D images. You can't make a 3D image of a flat image either. If there is no depth difference in the focus zone, you will never have 3D. With a microscope lens 4X (--> 40x via occulair) you already have enough images with small insects of 1 mm thickness. You can take the step size a bit smaller if necessary. The step size for the 4x lens may vary between 15 and 25 um.

The RoboPro fofware with the TXT-Controller is really an excellent solution. Especially the large PC screen makes it possible to add a lot of buttons, controls and readouts. I wrote the I2C lib for this and it is very simple because you only have to send a small data string each time. These strings contain data values or commands for the external I2C hardware FPGA chip. The run mode is a small loop that repeats the basic commands for 1 picture. Really this method of working is super good. If you don't have an FPGA I2C chip you can do this with a microcontroller with I2C as well.
My FPGA I2C chip can also perform all other functions at the same time. My 16 motors routines with quadrature decoders at high speed is still available, also the 32 PWM outputs for servos or 8 analog 12bit high speed inputs. For digital inputs and outputs I have 3 SPI lines that can control almost unlimited I/Os. This can all run parallel without having any influence on the execution times. Also multi tasking within RoboPro routine continues to work perfectly without interfering with each other. The I2C chip also has 7 super high speed inputs that can be sampled with 50MHz resolution. This allows me to measure very fast event signals or high precision times. All this can also work simultaneously during the microscope recordings.

Frans.

[fotoopa]

Here are some examples of the use of the TXT-Controller controlled microscope:

I took samples from a water basin to find small water creatures. After collecting, I selected them under a bino and applied them to a preprepared glass. Then placed under the microscope with the 4X lens --> 40x magnification. Here you see such an creature, photo frame is 3mm wide.


Original: https://www.flickr.com/photos/fotoopa_hs/48464742107

This creature was a bit too big to be fully photographed. The frame size here is 5.3mm.

Original: https://www.flickr.com/photos/fotoopa_hs/48463253296


Thanks to the stepper motor and the TXT-Controller with the I2C interface, the operation of the microscope is very comfortable and simple.


Frans.