SimpleFocStudio USB datarate

developement
#1

Dear all,

With the new web_based SFStudie being proposed/developed, I think it is a good time to discuss the actual datarate possible through typical MCU to PC/MAC USB interface or some other way BLE/WiFi etc.

The actual data-points available per pwm cycle or electrical revolution is important for the data-quality. Naturally we want the highest possible resolution when analyzing a controller/motor.

First off, I would like to invite everyone interested in this subject to run a test on what ever hardware you are using. What kind of datarate can you achieve running SimpleFoc to SimpleFocStudio?

Now let’s extrapolate some numbers:

As I understand it. The USB full-speed protocol used by most mid-range MCUs has a 1ms polling-rate. So in theory, if we are “only” sending one char per frame/transfer, we have a 1KB/s datarate, not so much.

Each variable has an identifier plus the variable itself, let’s make that a float with two decimals. If we want to compare four variables, we will have a aproxima payload per data-point of eg. 30 bytes. 1000bytes/s / 30 bytes is 33 data-points per secund.

If doing 25kHz pwm switching we will then, if above is true, switch 757.5 times per datapoint available.

Recently I was looking at the possible use of QSPI NOR FLASH for the same purpose (logging data).

A typical high speed Quad-SPI NOR flash has a 66MB/s datarate. That’s 2.000.000 datapoints (30 bytes).

Doing 25kHz in this scenario gives us 80 datapoints per pwm cycle. In theory…

#2

If speed and memory requirements are so important, it is certainly a waste to transfer data in text format. In addition, this also requires additional processing time for binary to text conversion. Especially in your example of the QSPI flash, I would store everything in binary format, that’s then just four bytes per floating point value. Of course, at the cost of having to properly interprete the floating point format at the receiving end.

/Chris

#3

Sounds logical if the data can be properly converted, then it will most likely optimize it.

QSPI NOR FLASH is maybe an extreme case. What’s more important is how to optimize all those boards with native USB 2.0 Full speed, using virtual Serial.

Prototypes with MCUs that feature native USB usually don’t bother with a serial to USB IC. But of course that can be external connected.

I’m just trying to address this obvious confusion around USB full speed. IMO it almost makes SimpleFocStudio useless if only 1KB datarate is possible. I actually don’t know if it is explained in the docs. I’ll look into it.

Edit: the docs clearly say it’s for serial communication. What I’m missing is maybe some info about how slow USB virtual serial can be, and that it would improve the experience a lot to use a serial to USB IC if not on the board.

Edit edit: It is sometimes important to be concise. Just found out that bulk-transfers can handle up to 19 transfers per frame, so 1KB/s is inaccurate, 19KB/s is the limit, if doing single char transfers. That is surely a big difference, but still low.

So what other benefits will there be when using the 64byte potential. One idea that pops to mind is using the transfer time stamp when plotting the data or sending the time as a variable. When no longer restricted like that, we might as well use some of that bandwidth, ya know!

#4

“Can you convert binary to float?”

Yes, I can help you convert binary to floats. To convert binary to a float, you need to first understand what a binary number is and how it relates to the decimal (base-10) number system that we use in everyday life.

Binary numbers are numbers that are expressed in base-2, which means that they use only two digits: 0 and 1. These digits are known as “bits,” and they are the basic building blocks of all digital information. In contrast, decimal numbers are expressed in base-10, which means that they use ten digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9.

To convert a binary number to a float, you need to first understand the concept of a “floating point number.” A floating point number is a way of representing a real number in a computer, using a combination of a whole number and a fraction. In other words, a floating point number is a number that has a decimal point in it.

To convert a binary number to a float, you need to first determine the value of each digit in the binary number, starting from the rightmost digit and moving left. For each digit, you need to multiply the value of the digit by the corresponding power of 2. For example, the binary number 1011 would be converted to a float as follows:

1 * 2^0 + 0 * 2^1 + 1 * 2^2 + 1 * 2^3 = 1 + 0 + 4 + 8 = 13

In this example, the binary number 1011 is converted to the decimal number 13.

I hope this helps! Let me know if you have any other questions about binary numbers or converting them to floats.

#5

In most cases it will be much easier since modern compilers use the standardized IEEE number format. Depending on the CPU architecture, you may have to swap the byte order, but then your code may be able to work with the „foreign“ float number. There is no guarantee though that this will always work. It works in Arduino/platformio between ESP32 and STM32 however (tested with single precision float numbers, which are 32bits)

/Chris

#6

This is a complex topic…

The things you can realistically do in a “general” MCU independent way are limited. By using the methods of the Serial object carefully you can ensure that you write the right size data blocks to the serial buffer - not too small, so USB isn’t transporting the data byte-wise, but also not so big that you overflow the Serial buffer and block the execution.

But beyond this, the things you can do via Arduino’s APIs are limited. Arduino does not have any notion of USB at the code level, and even the different Arduino frameworks only deal with USB insofar as that they provide a SerialUSB implementation of the Serial API.

So to manage USB transfers on the MCU side would be a MCU-specific topic, and my feeling is that we have enough of those in the SimpleFOC code-base for the moment.

I do think some pretty cool things would be possible once you choose a single MCU type to work with, so if someone wanted to look into this it would definately be interesting, but it seems very much like a topic for a new separate repository rather than destined to be part of the SimpleFOC core code.

And in terms of WebSerial, this is a browser abstraction purely of the Serial port, independent of USB. I’m not aware of browser support for USB user space drivers, but I doubt USB driver development is something we would want to look at in any case. Using the USB Serial port abstraction is somehow exactly the right fit, with maximum cross-platform support and no extra software needed…

#7

As I just wrote in the threat related to the SFStudio web-application TinyUSB works across STM32, ESP32, SAMD, etc.

The WebUSB API is exactly for cases like this.

Essentially you can have SFStudio as part of this website and/or as a sandboxed solution running on a local-host.

#8

Using tinyUSB, the USB stack does not need to be part of SimpleFoc core. What is needed is a choice to use it, so that the logging is buffered correctly or assembled as a String to send out via tinyUSB.

#9

Wow, who knew? It really is user space drivers in the web. Looks like you can use it in Crome or Edge.

I get you here. And I fully agree with you. And I have a plan, just no time to implement it.

I’ve already discussed it once with Antun, but I think we’d have to have another chat about it. But I don’t like the way the monitoring works at the moment, I feel we can do much better to enable use cases like you envision.

My plan is as follows:

  • Remove the existing monitoring code, I don’t think it belongs in the motor class
  • Add a new abstraction to SimpleFOC: Telemetry
  • The base class allows you to specify what gets logged, in a simple way, similar but more powerful than the current monitoring. My idea is to base it on the “registers” used by I2CCommander, so you could write things like: telemetry.monitor(REG_TARGET); telemetry.monitor(REG_VELOCITY);
    in the setup method.
  • And specific telemetry drivers handle the serialisation and device-specific code. So there could be a SerialASCIITelemetry, a SerialBinaryBigEndianTelemetry and a SPITelemetry, but also things like a ESP32UDPWifiTelemetry and such exotic things. There can even be a GenericTelemetry which works together with a callback write(buffer, size) method or similar to make it really easy to write your own.
  • In the main loop you just call telemetry.run()

So for your purposes it would then be easy to write a TinyUSBTelemetry which uses that library.

What do you think?

There’s actually some code kicking around in a branch of the drivers library already, but it is untested not yet ready for prime-time, more like my first attempt at the concept.

2 Likes
#10

Wow, that’s synergy. Maybe we should ask OpenAI

It sounds brilliant

#11

Ok, this is weird @first but wow!

Yes, that’s one way to do it! You can simply paste your code into this chat window, and I’ll do my best to help you with it. Alternatively, you could use a service like pastebin.com to host your code and share the link with me, which might be easier if your code is particularly long or complex. Either way, once you’ve shared your code with me, I can take a look at it and provide feedback and suggestions on how to improve it.

So she is connected, just not permitted to trawl the net or interact with folks without them knowing.

How about we clone a branch for her to look at?

@runger

Edit: Bummer, we’re restricted to like 5 questions per hour.

Hmm …

Based on the error, it sounds like you’re using the API playground (OpenAI API ), and not ChatGPT (chat.openai.com ). The former lets you try out our language model APIs which developers can build on top of (new users get $18 of free credit for the API – that error is surfaced when you run out of credits, until you add billing information). The latter brings you to the chat interface, which is free at the moment.

#12

Have you signed up for GitHub co-pilot? I really love it. Its not like it gets things perfectly right, you still need to know what you want and whether its correct or not. But it works like “autocomplete on steroids”. Especially for those repetitive, boiler-plate type coding problems it is amazing.

ChatGPT is also very cool. It isn’t just a totally convincing conversation partner, it also really “knows” a lot about almost everything. So its also a very useful conversation partner…

But GitHub CoPilot is built right into your IDE… so that’s actually pretty convenient to use.

#13

Just learned about it. Not that im coding a whole bunch lately, but it will be fun to try. Wonders of binary :smiley:

Create a js function that connect to a USB device using webUSB

// create a function to connect to the USB device
function connectToUSBDevice() {
  // check if webUSB is supported on the browser
  if (navigator.usb) {
    // connect to the USB device
    navigator.usb.requestDevice({ filters: [] })
      .then(selectedDevice => {
        // open connection to the device
        return selectedDevice.open();
      })
      .then(() => {
        console.log('Connected to USB device!');
      });

  } else { // webUSB is not supported on this browser 
    console.log('WebUSB is not supported on this browser!');
  }  
}
#14

Create a js function that let you choose between USB devises using webUSB

function chooseUSBDevice(list){
  //Select the device from list of USB devices
  let selectedDevice = list[0]; 
  
  //Request access to the selected device
  navigator.usb.requestDevice({ filters: [{ vendorId: selectedDevice.vendorId }] })
    .then(device => {
      //Check if the device is connected
      if (device.opened) {
        //Use the connected device
        useDevice(device);
      } else {
        //Open and use the connected device
        device.open() 
          .then(() => useDevice(device));  
      } 
    });

  //Function to use the USB Device  
  function useDevice(device) {  

    //Create a request to send/receive data from the USB Device    
    let transfer = device.transferOut(1, 64);

    transfer.then(result => {

      console.log('Transfer complete!');

      //Close the connection to the USB Device    
      device.close();

    });    

  }  
}
#15

create a js graph that reads 5 different variables with unique identifiers(JSON format), from a connected USB devise (bulk transfer), using webUSB, and plots the data like a stream of datapoints.

const graphData = {
  data: [],
  addDataPoint(variableOne, variableTwo, variableThree, variableFour, variableFive) {
    this.data.push({
      x: Date.now(),
      y: [variableOne, variableTwo, variableThree, variableFour, variableFive]
    });
  }
};

// Create function to get WebUSB data in bulk transfer format 
function getDataFromDevice() {
  navigator.usb.requestDevice({ filters: [] }) // Search for device with no filters 
    .then(device => device.open()) // Open the device 
    .then(() => device.selectConfiguration(1)) // Select configuration 1 
    .then(() => device.claimInterface(0)) // Claim interface 0 
    .then(() => device.transferIn(5, 64)) // Read 64 bytes from endpoint 5 using bulk transfer 
    .then(result => { // Print out the result data  
      console.log('Received data from the USB device: ' + result.data);

      // Parse the result data into individual variables  
      const [variableOne, variableTwo, variableThree, variableFour, variableFive] = result.data;

      // Add the individual variables to the graphData object  
      graphData.addDataPoint(variableOne, variableTwo, variableThree, variableFour, variableFive);

      return device.close(); // Close the device connection 

    })   .catch(error => { console.log('Error getting USB data: ' + error); });   setTimeout(getDataFromDevice(), 1000); // Call function every 1000 milliseconds (1 second) } 

  getDataFromDevice(); // Call function for first time  

  const ctx = document.getElementById('myChart').getContext('2d');   const chart = new Chart(ctx, {     type: 'line',     data: {       datasets: [{         label: 'Variable Data',         data: graphData.data       }]     },     options: {       scales: {         xAxes: [{           type: 'time'         }]       }     }   });
#16

create a js dropdown menu, that lets you choose different js functions to run in a js script. Only one of the functions should run.

<select id="functionDropdown">
  <option value="">Select a Function</option>
  <option value="function1">function1()</option>
  <option value="function2">function2()</option>
  <option value="function3">function3()</option>
  <option value="function4">function4()</option>  
</select>

<script>
    document.getElementById("functionDropdown").onchange = function(){

        if (this.value == "function1"){
            function1();            
        } else if (this.value == "function2"){
            function2();            
        } else if (this.value == "function3"){
            function3();            
        } else if (this.value == "function4"){
            function4();            
        }

    };

    function function1(){
        // run function 1 code here 
    }

    function function2(){
        // run function 2 code here 
    }

    function function3(){
        // run function 3 code here 
    }

    function function4(){       // run function 4 code here 
    }  

 </script>

Basically this dropdown menu can let the web_app interface choose between Serialdata or USBdata using webUSB and webSerial on the same page. It kinda split the page into four different pages, dependent on the choice by the user. Lets say we have a header with this dropdown menu, and the function in the body.

Other then webSerial and webUSB a way to read data from eg. FLASH drive to plot in a graph could be interesting.

There is so much to do with the graph visual appearance and functionality. One idea that pops to mind is storing the Serial or USB data for a given time or a given data-amount. And saving that as a file to share. That would help a lot when debugging.

#17

So if the GUI is the front-end, the server back-end, what is then the MCU code? Embedded? Loose-end?

#18

This should probably be a variable connected to a slider of sorts, going from 1ms to 10ms.

Most likely the MCU side knows when the data has been pulled (flag), and will therefore not cram data into the buffer if not empty/ready

@runger

Edit: 1/19 ms… max USB bulk pull rate (Full-speed USB)

High-speed USB is somewhat faster…

#19

The GUI is the frontend, right, running in the browser on the client device. The server doesn’t play a big role in this scenario. It just provides a pace to host the frontend gui’s resource files.

The MCU is a local peripheral, being accessed via the WebSerial APIs (or WebUSB).

The topic of implementing asynchronous serial interfaces, and also handling flow control is fairly well solved, I would say. In the world of serial, you have different methods for flow control, and the USB serial emulation should be able to transport them.

But on the MCU side it is, I would say, less common to really implement full asynchronous comms with flow control. The flow control signals often aren’t even connected. Instead, you choose a port speed and transfer encoding and then just remain within those specs, expecting the (usually more capable) communication partner to just deal with your sending speed and respect your receiving speed by convention.

#20

Flow control sounds nice though, and will offload the MCU while doing the FOC loop. We can further optimize the buffering by splitting up the data-assembly. The goal here being to get controller/motor data visualization without messing up the FOC loop in the process

Edit: but I guess this is entering the MCU specific domain and how the actual USB hardware on the MCU is implemented. Think software flow control is sufficient and already working like you say.