Using CloudLocate with u-blox combo modules

Get a position using a combo module 

In this section, you'll find out the reference implementation in Python for getting MEASX message from combo module and use it to get a position estimation from CloudLocate service in a power optimized way. You can implement the same logic in the host processor of your device in your preferred language. The code allows you to set some parameters to find the best trade off between power consumption and location accuracy. 

The script collects real-time the MEASX messages from a combo module and sends the raw measurements to CloudLocate when the configured conditions are met.

Before presenting the whole script (which you can run as-is, provided you have Combo Module EVK plugged in), let's have  look at some key sections of it:

1. Custom parameters to optimize the quality of MEASX message

COMPORT = "COM52"

BAUD_RATE = 115200

GNSS_TYPE = "GPS" 

TIMEOUT = 20

CNO_THRESHOLD = 25

MIN_NO_OF_SATELLITES = 4

MULTIPATH_INDEX = 1

PSEUDO_RANGE_ERR = 10

EPOCHS = 1

SEND_BINARY = True

FALLBACK_METHODOLOGY = FallbackConfig.FALLBACK_DO_NOT_SEND

where:

• COMPORT is the port on which you've connected the EVK

• BAUD_RATE is the rate on which your EVK is configured. In order to see what is your EVK's baud rate see the corresponding  products data sheet.

• GNSS_TPYE is the preferred constellation that will be used for the target MEASX messages. The script will ignore any satellite that does not belong to the configured constellation.  Our tests shows that the best performance can be achieved with GPS, but this can change from region to region. Only one constellation is allowed.

• TIMEOUT is the maximum number of seconds the script will try to get MEASX messages that fall under given criteria. 

• CNO_THRESHOLD: is the carrier-to-noise ratio threshold. Any satellite with a CNO below this threshold will be ignored.  The higher the value, the better the message. A meaningful threshold can be 35 dB/Hz, but the correct one can be achieved only by doing some testing in your real environment

• MIN_NO_OF_SATELLITES: each MEASX message can contain information for more than 1 satellite. This parameter allows you to set the number of satellites that are part of a MEASX message. The minimum value is 4; a low number could affect the accuracy, a high number (i.e. 32)  affects the size of the payload and the time to meet this condition  (and the risk to not meet the condition) and so finally the power consumption. An acceptable range for constrained devices can be between 4 and 10

It's worth noting that a MEASX message comprises 44 bytes for the header plus 24 bytes for each satellite.

• MULTIPATH_INDEX: multipath index determines the path that the GNSS signal took to reach the receiver (whether it had a straight line of sight, or it bounced off buildings or other obstacles). The lower value the better (3 can be used as starting point for testing, however 1 is suggested)

• PSEUDO_RANGE_ERR:  this parameter determines pseudo-range RMS error value. It shows the error index of the pseudo-distance between the satellite and the GNSS receiver.  

• EPOCHS: each MEASX message is 1 epoch. This parameter lets you configure the number of epochs to record (which fall under above parameters) before sending the payload to CloudLocate. The recommended range is between 1 to 3 epochs. This setting affects the size of payload and of course in several use cases the only acceptable value is 1.

• SEND_BINARY: set this flag when you want to send the raw measurement as a binary messages  (exactly as provided by GNSS) to CloudLocate service endpoint. Otherwise, Base64 encoded JSON payload will be sent. JSON format is the only option that you can use whenever you need to send delayed messages, because you need to add also date and time. A delayed message is a raw measurement that reaches CLoudLocate endpoint more than 59  seconds after the generation, useful for example if you do not have temporary the connectivity and you need to store the measurement in the device for post-processing. 

• FALLBACK_METHODOLOGY controls the way the script behaves if it does not find required MEASX messages as per main configuration. There are a few fallback methodologies defined in the code, and which one to use depends on this parameter. Available fallback methodologies (configurations) are:

  • FALLBACK_DO_NOT_SEND will not use any fallback, and is the default when running this script.

  • FALLBACK_NO_OF_SATELLITIES_ONLY will fallback to only match # of satellites (irrespective of CNO value).

  • FALLBACK_EXTEND_TIMEOUT will extend the timeout value, so if a match is not found without original time, it will add this more seconds.

2.   AT commands are used to communicate with the combo module. Given below is the code snippet of helper functions for sending AT commands on UART and parsing responses: 

def get_urc_param(urc, response, param_index):

    return response.split(urc)[1].split('\r\n')[0].split(',')[param_index].strip()

def wait_for_urc(serial_handler, wait_for, timeout=AT_COMMAND_TIMEOUT, terminate=True):

    serial_handler.timeout = timeout

    print(f"Waiting for URC {wait_for} with timeout value: {timeout} seconds")

    resp = serial_handler.read_until(bytes(wait_for, 'utf-8')).decode('utf-8')

    serial_handler.timeout = 0.1

    resp += serial_handler.read_until(bytes('\r\n', 'utf-8')).decode('utf-8')

    print(f"Received: {resp}")

    if resp.find(wait_for) == -1:

        print(f"Unable to receive {wait_for} from the module")

        if terminate:

            print("Terminating script!")

            exit()

    return resp

def send_at_command(serial_handler, command, wait_for='OK\r\n', timeout=AT_COMMAND_TIMEOUT, terminate=True):

    serial_handler.timeout = timeout

    is_binary = False

    if type(command) is bytearray:

        serial_handler.write(command)

        is_binary = True

    else:

        serial_handler.write(bytes(command + AT_COMMAND_SUFFIX, 'utf-8'))

    print(f"Writing to serial: {command}")

    resp = serial_handler.read_until(bytes(wait_for, 'utf-8')).decode('utf-8')

    if not is_binary:

        resp = resp.replace(command + "\r" + AT_COMMAND_SUFFIX, '')

    print(f"Received response: {resp}")

    if resp.find(wait_for) == -1:

        print(f"Unable to receive {wait_for} from the module")

        if terminate:

            print("Terminating script!")

            exit()

    return resp

Function send_at_command does the following tasks: 

2. • It takes AT command as an input, appends command's terminating characters to it and writes it to the serial channel open with the module.

   • It waits for the string passed as the parameter for the given timeout value. 

Function  wait_for_urc  waits for URC passed as parameter with the timeout value. 

Function  get_urc_param parse the URC received from the module and return the requested parameter index.

3.  Once you have adjusted the parameters, you can capture real-time MEASX messages using the following code snippet:

    ser = serial.Serial(COMPORT, BAUD_RATE, timeout=AT_COMMAND_TIMEOUT)

    # SEND AT to check module type number

    resp = send_at_command(ser, "AT+CGMM")

    TYPE_NUMBER = ("SARA-R4" if resp.find("SARA-R4")>=0 else "SARA-R5")

    print(f"Device TypeNumber: {TYPE_NUMBER}")

    # Check GNSS status if it is powered off turn it on

    resp = send_at_command(ser, "AT+UGPS?")

    if resp.find("+UGPS: 1") >= 0:

        send_at_command(ser, "AT+UGPS=0")

    send_at_command(ser, f"AT+UGPS=1,0,1")

    gpsTimer = time.time()

    print("GPS powered on")

    startTime = time.time()

    # continue reading MEASX messages until:

    # [a] timeout happens

    # [b] you get number of messages as per configuration

    # [c] save this measx message, along with satellite info so it can be used for fallback logic

    usedExtendedTime = False

    while (time.time() - startTime) <= TIMEOUT + extendedTime and validMessageCounter < EPOCHS:

        if time.time()-startTime>TIMEOUT:

            usedExtendedTime = True

        # read message from receiver

        resp = send_at_command(ser, "AT+UGUBX=\"B5 62 02 14 00 00 16 44\"", timeout=5)

        rawMessage = resp.split("\"")[1]

        rawMessage = binascii.unhexlify(rawMessage)

        # Removing MEASX HEADER from the message

        rawMessage = rawMessage.replace(MEASX_HEADER, b"")

        # calculate size of payload contained inside read MEASX message

        size = (rawMessage[1] << 8) | rawMessage[0]

        # checking size for a valid measx message

        if (size <= 33):

            print(f"Message skipped: {rawMessage.hex()}")

            # skipping this message

            continue

        # extract actual MEASX payload (without header, and checksum)

        measxMessagePayload = rawMessage[2:size + 2]

        # get the number of satellites contained in this message

        numSv = measxMessagePayload[34]

        print("Number of satellites: ", numSv)

        # Saving message in a proper structure

        processedMeasxMessage = {

            'measxMessage': rawMessage[0:size + 4],

            'maxCNO': 0,

            'satellitesInfo': {}

        }

        satelliteCount = 0

        # for the number of satellites contained in the message

        # we need to see if every satellite's data falls as per our configuration

        # because a single MEASX message can contain more than one satellite's information

        for i in range(0, numSv):

            # only accept the message if it fulfills our criteria, based on configuration parameters above

            gnss = measxMessagePayload[44 + 24 * i]

            if gnss == CONSTELLATION_TYPES[GNSS_TYPE]:

                # carrier-to-noise ratio

                cNO = measxMessagePayload[46 + 24 * i]

                psuedoRange = measxMessagePayload[65 + 24 * i]

                # save maximum CNO value as a separate key in stored measx message

                if cNO > processedMeasxMessage['maxCNO']:

                    processedMeasxMessage['maxCNO'] = cNO

                multipathIndex = measxMessagePayload[47 + 24 * i]

                svID = measxMessagePayload[45 + 24 * i]

                processedMeasxMessage['satellitesInfo'][svID] = {'cno': cNO, 'mpi': multipathIndex}

                if cNO >= CNO_THRESHOLD and multipathIndex <= MULTIPATH_INDEX and psuedoRange <= PSEUDO_RANGE_ERR:

                    satelliteCount = satelliteCount + 1

                    print(f"gnss: {gnss} ... svID:{svID} ... cNO: {cNO} ... multipathIndex: {multipathIndex} ... psuedoRange:{psuedoRange}")

        # saving processed message

        READ_RAW_MEASX_MESSAGES.append(processedMeasxMessage)

        if satelliteCount >= MIN_NO_OF_SATELLITES:

            MEASX_MESSAGE.extend(MEASX_HEADER)

            MEASX_MESSAGE.extend(bytearray(rawMessage[0:size + 4]))

            validMessageCounter = validMessageCounter + 1

The above code snippet does the following tasks:

• It opens up a COM port to communicate with the module.

• It  checks module's type number using "AT+CGMM" command. 

• It then sends an AT command to check power status of the GNSS chip on the EVK. If the chip is powered 'OFF', it turns its power 'ON'.  

• It sends a message to receiver and polls for the desired response.

• It then reads a chunk of data received from the receiver (based on the MEASX message header)

• It then determines, based on set parameters, if the read MEASX message falls under the required criteria to decide if keeping or discarding it.

  • It also saves this MEASX message in memory, to be used by fallback logic.

• At the end, the code determines if the desired MEASX message is available or not. If not, the code looks if any fallback methodology has been set (not shown in code snippet above) and applies the selected fallback configuration against saved MEASX messages.

4. After capturing  a real-time MEASX message, you need to send it to CloudLocate to get the position. For this, as mentioned in the CloudLocate getting started, you should already have created a CLoudLocate thing.  The following code snippet establishes an MQTT connection with the server, sends the data, and gets the position back on the device. It uses the MQTT stack of the module.

# Prepare message according to configuration

    if SEND_BINARY:

        # Sending RAW binary message

        MQTT_MSG = MEASX_MESSAGE

    else:

        # convert our MEASX byte-array into base64 encoded string

        BASE64_ENC_PAYLOAD = base64.b64encode(MEASX_MESSAGE).decode()

        MQTT_MSG = f"{{\"body\": \"{BASE64_ENC_PAYLOAD}\"}}"

        print(f"JSON formatted payload: {MQTT_MSG}")

    if len(MQTT_MSG) > 1024:

        print(

            "Cannot send MQTT message greater than 1KB. Please reduce the number of EPOCHS in configuration parameters to reduce the size.")

        exit()

    send_at_command(ser, "AT+CEREG=1")

    resp = send_at_command(ser, "AT+CFUN?")

    cfun_mode = int(resp.split("+CFUN:")[1].split("\r\n")[0].split(',')[0].strip())

    if cfun_mode != 0:

        send_at_command(ser, "AT+CFUN=0", timeout=5, terminate=False)

    send_at_command(ser, "AT+CFUN=1", timeout=5, terminate=False)

    resp = wait_for_urc(ser, "+CEREG:", timeout=5)

    cereg_stat = int(get_urc_param("+CEREG:", resp, 0))

    if cereg_stat == 2:  # module is trying to attach

        resp = wait_for_urc(ser, "+CEREG:", timeout=5)

        cereg_stat = int(get_urc_param("+CEREG:", resp, 0))

    # Accepted value is 1 or 5 (Home registration or Roaming registration)

    if cereg_stat != 1 and cereg_stat != 5:

        print(f"Could not register to network, CEREG URC: {resp}. Terminating script")

        exit()

    # Activating cellular module PDP context

    send_at_command(ser, "AT+UPSD=0,0,0", terminate=False)

    send_at_command(ser, "AT+UPSD=0,100,1", terminate=False)

    if TYPE_NUMBER == "SARA-R5":

        send_at_command(ser, "AT+UPSDA=0,3", terminate=False)

        wait_for_urc(ser, "+UUPSDA:")

    # Setting Up module MQTT client:

    send_at_command(ser, f"AT+UMQTT=2,\"{Hostname}\"")

    send_at_command(ser, f"AT+UMQTT=4,\"{Username}\",\"{Password}\"")

    send_at_command(ser, f"AT+UMQTT=0,\"{DeviceID}\"")

    # connecting to MQTT broker

    send_at_command(ser, "AT+UMQTTC=1")

    resp = wait_for_urc(ser, "+UUMQTTC:", timeout=30)

    mqtt_client_status = int(get_urc_param("+UUMQTTC:", resp, 1))

    if mqtt_client_status == 0:

        # Retrying MQTT client connection

        send_at_command(ser, "AT+UMQTTC=1")

        wait_for_urc(ser, "+UUMQTTC: 1,1", timeout=30, terminate=True)

    # Subscribing to TOPIC on which position will be returned

    send_at_command(ser, f"AT+UMQTTC=4,2,\"{MQTT_SUB_TOPIC}\"", f"+UUMQTTC: 4,1,2,\"{MQTT_SUB_TOPIC}\"", timeout=5)

    # Publishing message

    send_at_command(ser, f"AT+UMQTTC=9,2,0,\"{MQTT_PUB_TOPIC}\",{len(MQTT_MSG)}", ">")

    send_at_command(ser, MQTT_MSG, "+UUMQTTC: 6,1", 30)

    # Reading incoming message

    resp = send_at_command(ser, "AT+UMQTTC=6,1")

    json_resp = "{" + resp.split("{")[1].split("}")[0] + "}"

    json_resp = json.loads(json_resp)

    lat = json_resp["Lat"]

    lon = json_resp["Lon"]

    print(f"I am here: https://www.google.com/maps/search/?api=1&query={lat},{lon}")

    if TYPE_NUMBER == "SARA-R5":

        print("Deactivating PDP context ")

        send_at_command(ser, "AT+UPSDA=0,4")

    send_at_command(ser, "AT+CFUN=0")

The above code snippet performs the following tasks: 

4. • If SEND_BINARY flag is set to TRUE, it converts MEASX message to binary, else MEASX is converted to JSON formatted Base64 string.

   • It turns on the module radio and wait until the module gets registered to network.

   • Once the module gets registered to network, it then activates the PDP context

   • It then configures the MQTT client with server name, username, passwords and device ID.

   • Once the connection to the broker is established, it subscribes to the specific topic. 

   • It then publishes the MEASX message and waits for the position. 

Note: the whole script reported here is available in the Download section. Do not forget to:

4. • set the MQTT credentials for the Location Thing in the proper section

   • configure the communication port, and baud rate

   • adjust the settings

Note: the code is provided just as reference and you are allowed to modify it accordingly to your needs

Reducing the power consumption

The reference implementation has been designed to optimize as much as possible the power consumption, to address those scenarios where device shall stay in the field for several years without any battery recharge. The result has been realized by

• switching the GNSS chip ON only when required to get the raw measurement. When the target measurement has been found (accordingly to the desired settings) the GNSS is immediately switched OFF

• during the GNSS scan, the communication between the GNSS and the cellular chips is reduced to the minimum required, avoiding unnecessary communications.

• the cellular radio is switched off until you do not need to send the measurement to the service endpoint. Once the message has been sent, the radio is switched off again

Nevertheless it is suggested to look at  cellular PSM and deep sleep mode to further reduce the power consumption. This is a global setting that affects all the communications with the cellular network that your use-case requires, and not only CloudLocate.

It's worth noting that PSM is not supported by all MNO and could not be supported when you are in roaming condition due to lack of agreement between the MNOs;  using the proper configurations the cellular module is able to detect if PSM is supported and adjust the behavior accordingly. More information can be found at Section 8 of [8] and [9]