Slow Serial Communication through UNO and Raspberry Pi using PySerial

I have been trying to get fast, reliable serial communication from raspberry pi to Arduino (read and write) and I have no success. I have it running but its not fast whatsoever, it instantly writes to the Arduino, but reading takes about 3 seconds... I'm hoping to get it down to 100-400 ms, My code is below....

If I don't sleep in the while loop, the data will never be received...

import serial
import time
import sys


ser = serial.Serial("/dev/ttyACM1")  # open first serial port
ser.baudrate = 9600
#ser.open()

time.sleep(1)
ser.setDTR(level=0)
time.sleep(1)

ser.write(sys.argv[1])
time.sleep(1)
print(ser.readline())

ser.close()
void setup() {
  Serial.begin(9600);
}


void loop() {

  String msg = "";
  if (Serial.available() == true) {
    //We take in a string msg from Serial
    msg = Serial.readString();
    //Remove the duplicate spaces

    //We then break it up into a program and args
    String response = doSomething(msg)
    Serial.println(response);
    delay(100);
  }

A couple of things.

On the Arduino side Serial.readString() is a blocking function that waits for data to arrive. Have a look at the examples in Serial Input Basics - simple, reliable. non-blocking ways to receive data.

On the Python side it seems that you open the serial port for every message. That will cause the Arduino to restart and, even if that is not itself a problem, it will mean you have to wait for the reset process to complete.

Have a look at this Python - Arduino demo

Also, though a lesser issue, use a higher baud rate. 500,000 baud should work fine.

...R

So in your opinion, what do you believe to be the largest bottle-neck?

What I am doing is creating a car that uses an Arduino to control the motors and sensors, the pi runs on java and invokes python to communicate to the arduino to retrieve sensor data and send the best move to the car. The reason I want the pi is to use it as a receiver for a remote control...

Thank you

You need to fix both of the things I mentioned.

Your description of your project is not clear.

  • How will the PI connect to the Arduino to get the data?
  • Will it be connected all the time?
  • If you are running a Java program on your PI why not use the JSSC library to communicate with the Arduino?

If you only want to connect to the Arduino occasionally a somewhat different technique will probably be appropriate - but then the latency would hardly be a problem.

...R

  • The pi will connect over USB
  • Yes it will be connected all the time, however I need the pi to write to serial then read from serial to get the Arduino's response.
  • Will I be able to get jSSC running without an IDE? I would like to keep this as minimal as possible, any suggestions?

ConditionalCoder:

  • Yes it will be connected all the time, however I need the pi to write to serial then read from serial to get the Arduino's response.

I don't understand your use of the word "however". Why would the fact that you are connected all the time prevent you from occasionally requesting data? I reckon it will make it easier.

  • Will I be able to get jSSC running without an IDE? I would like to keep this as minimal as possible, any suggestions?

I don't understand why you link JSSC with an IDE - it is just a more modern equivalent of RxTx. JSSC will not require you to use an IDE - and I am not even sure what you mean by "an IDE".

...R

I used 'however' because I was unsure if you were thinking I was just listening to the Arduino and never writing to it...

I haven't seen a way to install jSCC without a GUI involved. I'd like to keep this in the terminal since I don't have any monitors...

ConditionalCoder:
I haven't seen a way to install jSCC without a GUI involved.

Post a link to the place where you are getting that information.

It's quite some time since I used JRuby and I am rusty. But I certainly used both RXTX and JSSC without any GUI code.

This looks like a more complete version of the JRuby program - maybe it will make things clearer. to be honest I have forgotten how to get it to work and I used to be very familiar with it.

# 10 Mar 2014

# this is designed to work with ... ArduinoPCa4e.ino ...

# this receives data after a 254 followed by the number of bytes followed by the data followed by 255
#   if the number of bytes is 0 it assumes a debug string

# this version uses JSSC - Java Simple Serial Connector - rather than RxTx 
#     RXTX seems to have disappeared from the web
#   apparently all the necessary binaries are included in the Jar
#   note that JSSC returns a signed java byte -  hence & 0xFF to convert to an unsigned value
#   also note the use of .to_java_bytes to convert the JRuby string into a Java byte array


def waitForArduino

    # wait until the Arduino sends something - allows time for Arduino reset
   msg = ""
   while msg.include?("Arduino Ready") != true

      while $sp.getInputBufferBytesCount == 0
      end

      # then wait until an end marker is received from the Arduino to make sure it is ready to proceed
      x = 32
      while x != $endMarker # gets the initial debugMessage
        x = $sp.readBytes(1)[0] & 0xFF
	      msg = msg + x.chr

      end

      displayDebug msg
      
      puts
    end
    
end

#=====================================

def sendToArduino(sendStr)

	txLen = sendStr.size.chr
	adjSendStr = encodeHighBytes(sendStr)
	adjSendStr = $startMarker.chr + txLen + adjSendStr + $endMarker.chr
	$sp.writeBytes(adjSendStr.to_java_bytes)
	
end

#======================================

def recvFromArduino

	ck = ""
  x = 32 # any value that is not an end- or $startMarker
  byteCount = -1 # to allow for the fact that the last increment will be one too many

    # wait for the start character
  while  x != $startMarker 
    x = $sp.readBytes(1)[0] & 0xFF
  end
  
    # save data until the end marker is found
  while x != $endMarker
    ck = ck + x.chr 
    x = $sp.readBytes(1)[0] & 0xFF
    byteCount += 1
  end
  
    # save the end marker byte
  ck = ck + x.chr
  
  returnData = []
  returnData[0] = ck[1].ord
  returnData[1] = decodeHighBytes(ck)
  
  return returnData

end

#======================================

def encodeHighBytes(inStr)
  outStr = ""
  inStr.size.times do |n|
    x = inStr[n].ord
    if x >= $specialByte
       outStr << $specialByte.chr
       outStr << (x - $specialByte).chr
    else
       outStr << x.chr
    end
  end
#  puts "encINSTR   #{bytesToString(inStr)}"
#  puts "encOUTSTR  #{bytesToString(outStr)}"
  return outStr
end

#======================================

def decodeHighBytes(inStr)
  outStr = ""
  n = 0
  while n < inStr.size
     if inStr[n].ord == $specialByte
        n += 1
        x = ($specialByte + inStr[n].ord).chr
     else
        x = inStr[n]
     end
     outStr << x
     n += 1
  end
#  puts "decINSTR   #{bytesToString(inStr)}"
#  puts "decOUTSTR  #{bytesToString(outStr)}"
  return outStr
end

#======================================

def displayData(data)

  n = data.size - 3

	puts "NUM BYTES SENT-> #{data[1].ord.to_s}"
	puts "DATA RECVD BYTES-> #{bytesToString(data[2, n])}"
	puts "DATA RECVD CHARS-> #{data[2, n]}"
end

#======================================

def bytesToString(data)
  byteString = ""
  n = data.size
	n.times do |s|
	  byteString << data[s].ord.to_s
	  byteString << "-"
	end
	return byteString
end

#======================================

def displayDebug(debugStr)
  n = debugStr.size - 3
  puts "DEBUG MSG-> #{debugStr[2, n]}"
end

#======================================

# the program starts here

$LOAD_PATH << Dir.pwd

require 'java'

require 'lib/jssc.jar' 
puts "JSSC found"
#java_import('gnu.io.CommPortIdentifier') 
#java_import('gnu.io.SerialPort') { 'JSerialPort' }
java_import('jssc.SerialPort')
java_import('jssc.SerialPortException')

at_exit{ $sp.closePort if $sp.isOpened }


$sp = SerialPort.new("/dev/ttyS80")
$sp.openPort
# $sp.setParams(57600, 8, 1, 0)
#$sp.setParams(230400, 8, 1, 0)
#$sp.setParams(250000, 8, 1, 0) # does not work
$sp.setParams(500000, 8, 1, 0)
#$sp.setParams(1000000, 8, 1, 0)


puts "Serial port opened"

$startMarker = 254
$endMarker = 255
$specialByte = 253


waitForArduino
puts "Arduino is ready"

testData = []
testData[0] = "abcde"
testData[1] = "zxcv1234"
testData[2] = "a" + 16.chr + 32.chr + 0.chr + 203.chr
testData[3] = "b" + 16.chr + 32.chr + 253.chr + 255.chr + 254.chr + 253.chr + 0.chr
testData[4] = "fghijk"

numLoops = testData.size

n = 0
waitingForReply = false
while n < numLoops

	teststr = testData[n]


	if $sp.getInputBufferBytesCount == 0 && waitingForReply == false
	  sendToArduino(teststr)
	  puts "=====sent from PC=========="
	  puts "LOOP NUM #{n}"
	  puts "BYTES SENT -> #{bytesToString(teststr)}"
	  puts "TEST STR #{teststr}"
	  puts "==========================="
    waitingForReply = true
#	  n += 1
	end
	
	if $sp.getInputBufferBytesCount > 0
    dataRecvd = recvFromArduino

    if dataRecvd[0] == 0
      displayDebug(dataRecvd[1])
    end

    if dataRecvd[0] > 0
      displayData(dataRecvd[1])
      puts "Reply Received"
      n += 1
      waitingForReply = false
    end

    puts
    puts "==========="
    puts
  end

  sleep 0.3

end # while n < numLoops


$sp.closePort

I am only posting this so you get a general idea of how it uses JSSC.

...R