ESP32-S3 partition tables and optimizing memory for Arduino IDE 2.x

Development Tools IDE 2.x
1

I published instructions on creating customize memory partition maps for ESP32-S3 in Arduino IDE 2.x to Reddit. This is in support of my Reflections open-source project for creating entertaining mobile experiences. It's up at:

https://www.reddit.com/r/esp32/comments/1gnlp9h/esp32s3_partition_tables_and_optimizing_memory/

2

If it is a custom flash memory partition table for a single project you should be able to simply name it partitions.csv, copy it into the sketch folder and select custom partition from the tools menu of the Arduino IDE. Your description seems somewhat elaborate.

1 Like
3

I also think this would be a more user friendly approach.

Keep in mind that, if you take the approach of modifying boards.txt, then the modification is lost every time the user updates to a new version of the "esp32" boards platform.

In case you are curious about this feature of the platform, you can see how it is implemented here:

This is not necessary. If a file named partitions.csv is present in the sketch folder, it will always be used, regardless of what is selected from Arduino IDE's Tools > Partition Scheme menu.

2 Likes
4

I've learned something as well. Thank you for that.

2 Likes
5

Elaborate?! ha! It's about 15 steps more than I ever wanted. I didn't find anything about partitions.csv when I searched for a solution. Thank you for pointing this out. -Frank

6

I didn't find anything about partitions.csv when I searched for a solution. Thank you for pointing this out. -Frank

7

You are welcome. I'm glad if I was able to be of assistance.

Thanks also to you for sharing your project with the community under an open source license!

The feature of the "esp32" boards platform that allows defining a custom partitions scheme by adding the partitions.csv file to the sketch folder is documented here:

https://docs.espressif.com/projects/arduino-esp32/en/latest/tutorials/partition_table.html?highlight=partitions%20csv#using-a-custom-partition-scheme

1 Like
8

Has anyone tried this and had it work? Arduino IDE 2.3.4, 2.0.18-arduino.5 ESP32 Board package.
I've created partitions.csv in my project folder, wrote a quick program to read back the partition info, compiled and downloaded, and it doesn't seem to work.

9

What happens if, in the IDE tools menu, you set the core debug level to verbose ? Do you see evidence of your new partition settings when you start the ESP32 ?

10

I didn't see that verbose option.
But I did get it working I think! I used the esptool to program (not dfu) after some searching around, and I think it is now using the partitions.csv file in my project folder.

  1. started the board into ROM BOOT mode (short B1-GND on reset).
  2. Selected my board (Arduino Nano ESP32) manually, because it only identifies as "ESP32 family" on the USB serial port
  3. Selected tools->programmer->esptool.
  4. Then Sketch->Upload using programmer

Seems to work. I'll continue to test more.

11

My test code:


#include "esp_flash.h"
#include "esp_log.h"
#include "esp_partition.h"

#define FLASH_TEST_ADDRESS 0x0000  // Example address within a data partition (Sector (0x1000) aligned)

// SPI Flash parameters (adjust as needed for your specific flash chip)
#define FLASH_PAGE_SIZE 256     // Typical page size
#define FLASH_SECTOR_SIZE 4096  // Typical sector size

// the setup routine runs once when you press reset:
void setup() {
  Serial.begin(115200);

  delay(1000);  // WAIT FOR SERIAL MONITOR
}

// void loop runs over and over again
void loop() {

  static bool getp = false;
  static bool gotp = false;
  static bool write = false;

  if (Serial.available() > 0) {
    // read incoming serial data:
    char inChar = Serial.read();
    if (inChar == 'p')
      getp = true;
    if (inChar == 'w')
      write = true;
    
  }

  const esp_partition_t* data_partition;

  if (getp) {
    getp = false;


    /**********FLASH STUFF***********/

    data_partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_FAT, NULL);

    if (data_partition == NULL) {
      Serial.println("Data partition not found!");
      return;
    }
    Serial.print("Label of the partition: ");
    Serial.println(data_partition->label);
    Serial.print("Chip_ID of partition: 0x");
    Serial.println(data_partition->flash_chip->chip_id,HEX);
    Serial.print("starting address of the partition in flash: 0x");
    Serial.println(data_partition->address,HEX);
    Serial.print("size of the partition in flash: 0x");
    Serial.println(data_partition->size,HEX);
    // Serial.println("erase size of the partition: ");
    // Serial.println(data_partition->erase_size,HEX); // apparently not supported

    gotp = true;
  }

  if (gotp && write) {
    write = false;

    uint8_t write_data[FLASH_PAGE_SIZE];
    uint8_t read_data[FLASH_PAGE_SIZE];

    // Fill write_data with some test data
    for (int i = 0; i < FLASH_PAGE_SIZE; i++) {
      write_data[i] = i % 256;
    }

    // Erase the flash sector
    Serial.println("Erasing flash sector...");
    esp_err_t err = esp_partition_erase_range(data_partition, FLASH_TEST_ADDRESS, FLASH_SECTOR_SIZE);  // erase from the start of the partition.
    if (err != ESP_OK) {
      Serial.print("Flash erase failed: ");
      Serial.println(esp_err_to_name(err));
      return;
    }
    Serial.println("Flash sector erased.");

    // Write data to flash
    Serial.println("Writing data to flash...");
    err = esp_partition_write(data_partition, FLASH_TEST_ADDRESS, write_data, FLASH_PAGE_SIZE);
    if (err != ESP_OK) {
      Serial.print("Flash write failed: ");
      Serial.println(esp_err_to_name(err));
      return;
    }
    Serial.println("Data written to flash.");

    // Read data from flash
    Serial.println("Reading data from flash...");
    err = esp_partition_read(data_partition, FLASH_TEST_ADDRESS, read_data, FLASH_PAGE_SIZE);
    if (err != ESP_OK) {
      Serial.print("Flash read failed: ");
      Serial.println(esp_err_to_name(err));
      return;
    }
    Serial.println("Data read from flash.");

    // Verify the data
    bool data_match = true;
    for (int i = 0; i < FLASH_PAGE_SIZE; i++) {
      if (write_data[i] != read_data[i]) {
        data_match = false;
        break;
      }
    }

    if (data_match) {
      Serial.println("Data verification successful!");
    } else {
      Serial.println("Data verification failed!");
    }

  }

}

My partitions.csv:

Name, Type, SubType, Offset, Size, Flags

nvs, data, nvs, 0x9000, 0x6000,
phy_init, data, phy, 0xf000, 0x1000,
factory, app, factory, 0x10000, 0x50000,
test1, data, fat, 0x60000, 0x3a0000,

12

Your code looks right. Maybe Arduino IDE 2 is getting in the way?

Here is my partitions.csv I'm using with an ESP32-S3. I shared instructions on how I created it at See instructions at: ReflectionsOS/Docs/Partition tables and optimizing memory in Arduino IDE.md at main · frankcohen/ReflectionsOS · GitHub

Espressif ESP32 Partition Table

for Reflections project,

GitHub - frankcohen/ReflectionsOS: Reflections is a hardware and software platform for building entertaining mobile experiences.

Name, Type, SubType, Offset, Size, Flags

nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
ota_0,app,ota_0,0x10000,3712K,
ota_1,app,ota_1,0x3b0000,3712K,
eeprom,data,153,0x750000,4K,
spiffs,data,spiffs,0x751000,128K,
coredump,data,coredump,0x771000,64K,

1 Like