RedTide 650 support not working

[JuanCab]

spectrometer and system information

• model: RedTide 650
• operating system: macOS 10.12.5
• python version:3.5.1
• python-seabreeze version: 0.6.0
• installed-via: _conda

current problem

In a nutshell, the same code I have that works with an Ocean Optics USB2000 when applied to a RedTide 650 (which appears to be listed on the list of compatible devices) doesn't work due to a failure to connect to the spectrometer. The specific steps are below:

steps to reproduce

Specifically, although Apple's System Profiler clearly shows the RedTide connected (and I can access it via Vernier's Logger Pro software), when I do

import seabreeze.spectrometers as sb
devices = sb.list_devices()
print(devices)

I get a blank list.

[ap--]

With the new conda packages python-seabreeze defaults to the cseabreeze backend.
Maybe USB650 support doesn't work in cseabreeze... If so we need to file this upstream again.

In the meantime try installing python-seabreeze and pyUSB in conda:

conda install -c poehlmann python-seabreeze
pip install pyusb

(Note: the akode channel that we used before to install pyusb only ships it for 2.7. So we'll just install pyusb in conda via pip from pypi)

To check if pyusb was installed correctly run (it should not raise an exception):

import usb.core
usb.core.find()

and then:

import seabreeze
seabreeze.use("pyseabreeze")
import seabreeze.spectrometers as sb
devices = sb.list_devices()
print(devices)

I hope it'll work. I'll check in the meantime why the 650 isn't working with cseabreeze.

[JuanCab]

I will try this when I get back in the lab, but one quick question, is there a way to properly check if the cseabreeze interface works (for example, if devices = sb.list_devices() returns no devices) and then switch the interface to pyusb? I'm going to try an approach, but I suspect I can't simply re-import seabreeze.spectrometers.

[ap--]

Hmm. I think this might work on OSX and Linux:

# There is definitely a spectrometer connected, and the code
# should first try to connect via cseabreeze, and switch to
# pyseabreeze if it can't find the spectrometer
from seabreeze.backends import _use_cseabreeze
_lib = _use_cseabreeze()
if _lib is not None and _lib.device_list_devices():
    pass  # cseabreeze import works and it finds a spectrometer
else:
    # cseabreeze import failed or it didn't find a spectrometer
    import seabreeze
    seabreeze.use("pyseabreeze")

# continue normally
import seabreeze.spectrometers as sb
...

[ap--]

So as additional info:

I think reimporting seabreeze.spectrometers won't work,

but _use_cseabreeze and _use_pyseabreeze in seabreeze.backends return handles that have all the library methods including device_list_devices().
So it should be possible to do the check before importing seabreeze.spectrometers

[ap--]

Note to self: Maybe this should be default behavior...

[JuanCab]

Using your 'auto selection' code I was able to see the USB650, but I am getting errors when attempting to take a spectrum, even turning off dark and non-linearity corrections. The errors don't occur during every exposure, but happen about 80% of the time. Looks to be ultimately a USB overflow problem (based on reviewing error messages).

Juan

Traceback (most recent call last):
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 14, in decorated_func
    return func(*args, **kwargs)
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/spectrometer.py", line 87, in get_unformatted_spectrum
    timeout=int(self._INTEGRATION_TIME_MAX * 1e-3 + self.usbtimeout_ms))
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/communication.py", line 65, in usb_read_highspeed
    return self._device.read(self._ENDPOINT_MAP.highspeed_in, size, timeout=timeout)
  File "~/anaconda3/lib/python3.5/site-packages/usb/core.py", line 988, in read
    self.__get_timeout(timeout))
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
    timeout)
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 936, in __read
    _check(retval)
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 84] Overflow
Traceback (most recent call last):
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 14, in decorated_func
    return func(*args, **kwargs)
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/spectrometer.py", line 87, in get_unformatted_spectrum
    timeout=int(self._INTEGRATION_TIME_MAX * 1e-3 + self.usbtimeout_ms))
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/communication.py", line 65, in usb_read_highspeed
    return self._device.read(self._ENDPOINT_MAP.highspeed_in, size, timeout=timeout)
  File "~/anaconda3/lib/python3.5/site-packages/usb/core.py", line 988, in read
    self.__get_timeout(timeout))
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
    timeout)
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 936, in __read
    _check(retval)
  File "~/anaconda3/lib/python3.5/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 84] Overflow

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 14, in decorated_func
    return func(*args, **kwargs)
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/spectrometer.py", line 129, in get_formatted_spectrum
    self.get_unformatted_spectrum(tmp)
  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 23, in decorated_func
    raise SeaBreezeError(msg)
seabreeze.pyseabreeze.interfaces.common.SeaBreezeError: Error while reading raw spectrum.
Traceback (most recent call last):

  File "<ipython-input-17-710a9766b31a>", line 1, in <module>
    runfile('~/mycode/spec_controller.py', wdir='~/Documents/Research/Eclipse 2017/Experiments/Eclipse Software')

  File "~/anaconda3/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 880, in runfile
    execfile(filename, namespace)

  File "~/anaconda3/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "~/mycode/spec_controller.py", line 28, in <module>
    lamb, intens = spec.exposure(dev, time, darkcorr=False, nlcorr=False, verbose=True)

  File "~/mycode/speclib.py", line 134, in exposure
    intens = spec.intensities(correct_dark_counts=darkcorr, correct_nonlinearity=nlcorr)

  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/spectrometers.py", line 150, in intensities
    out[transfered_N:])

  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/wrapper.py", line 123, in spectrometer_get_formatted_spectrum
    return device.interface.get_formatted_spectrum(out)

  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 14, in decorated_func
    return func(*args, **kwargs)

  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/spectrometer.py", line 129, in get_formatted_spectrum
    self.get_unformatted_spectrum(tmp)

  File "~/anaconda3/lib/python3.5/site-packages/seabreeze/pyseabreeze/interfaces/common.py", line 23, in decorated_func
    raise SeaBreezeError(msg)

SeaBreezeError: Error while reading raw spectrum.

[ap--]

Does this also occur when you don't use the "auto-select" code? So just:

import seabreeze
seabreeze.use("pyseabreeze")
import seabreeze.spectrometers as sb

And possible ways to proceed are:

1. We fix the seabreeze C library (and therefore cseabreeze)
   1. File an issue on sourceforge, and request USB650 support (Because, while they provide windows drivers and correct udev-rules, it's not implemented in the code)
   2. I can walk you through how to do it yourself if you're comfortable with C/C++.
2. We fix pyseabreeze
   1. I can walk you through how to do it yourself.

Anyways, it would be good to have a datasheet with the full usb command set and compare it in detail with the one for the USB2000 (Because I think it's almost identical to that one)

[JuanCab]

Not using the auto-select does in fact stabilize things. I can now perform exposures on the USB650 without issue.

I am comfortable with C (its been a few years, but it dominated my grad school days), less so with C++. Feel free to provide me with guidance on how to fix this if by fix you mean allowing auto-select to function. I think I should file an issue with seabreeze as well, but do you have a specific error to report other than USB 650 support is not functional?

[ap--]

To just fix the auto-select, I think we don't need to touch any of the C/C++ stuff...

I think my previous approach doesn't work because the cseabreeze backend is not unloaded correctly.

Maybe try:

import usb.core
import seabreeze
if usb.core.find(idVendor=0x2457, idProduct=0x1014):
    # If we find a connected USB650
    seabreeze.use("pyseabreeze")
else:
    seabreeze.use("cseabreeze")

import seabreeze.spectrometers as sb

But: Why don't you just use the pyseabreeze backend for all spectrometers?

And regarding the upstream ticket: it's just that USB650 support is missing.

[JuanCab]

I didn't want to try the new backend because I am in the middle of an experiment and my mentality is to keep everything the same. That said, I can't imagine there should be a difference, so that may be the easier solution here.

I'll try your solution to the auto-switching shortly and if that fails I will just switch to using pyusb.

[JuanCab]

FYI, I posted a trouble ticket for seabreeze, response was that they intentionally left out RedTide USB650 support. See https://sourceforge.net/p/seabreeze/tickets/32/

[JuanCab]

On a positive note, your "switching" code above does seem to do the trick.

[JuanCab]

I take it back, I am still getting USB overflow errors with the RedTide USB650, even when switching to using pyUSB exclusively. :/

And I have also discovered when I do manage to access the RedTide USB650, I often get a uniform 2409.00 (2417.15893453) counts in every channel with non-linearity corrections turned off (on). Clearly not real data. This only appears to occur with the RedTide USB650, my Ocean Optics USB2000 appears to be behaving OK, even when accessed via pyUSB.

Something strange is going on here. Could there be some issue with how python-seabreeze communicates with the RedTide USB650? I was initially getting "real" looking data, but now its either overflowing or clearly bogus data.

[ap--]

Regarding the sourceforge ticket: Well I guess that explains why I can't find a USB-communication datasheet for the USB650... Weird though that they intentionally leave out support for it...

Regarding the overflow issue: Without the hardware or a datasheet it's hard for me to help with this.
It could be, that some of the assumed parameters for the 650 in pyseabreeze/interfaces/defines.py or in pyseabreeze/interfaces/init.py are wrong.

Is your measuring protocol doing some asynchronous stuff? Or something else that might cause the sporadic failures?

[JuanCab]

I'm not doing anything asynchronous. Just attaching the spectrometer, taking a reading, and detaching the spectrometer. And as I said, it works with the Ocean Optics USB2000 even when accessing via pyusb. I am doing this in iPython, so maybe iPython doesn't close something it should.