Project libraries and dependencies automatically to one package for export

The Arduino IDE has been a powerful tool for many years, but I was genuinely surprised to find that there is no built-in functionality to automatically export all the necessary files (such as libraries and dependencies) into a single project folder before compiling the code. So that you can have all libraries in same folder for export. Compiling gathers all the libraries from different folders but if you want to send an uncompiled version of the code it would be handy to have automatic package production option. Currently, users must manually locate and copy all the libraries and dependencies referenced in their sketch and transfer them to the folder containing the uncompiled .ino file if you wants to have all needed files in same folder.

This process can be quite tedious and time-consuming. A simple, one-button solution to bundle all required files into a single, portable project folder would greatly improve usability.

Even an enhancement to the existing library path viewer would be helpful. For example, when hovering over an #include <libraryname.h> directive, the IDE displays the library path. If there were an option to directly copy the library or its path to the project folder from this interface, it would streamline the process significantly.

I believe these changes would save a lot of time and make the Arduino IDE even more user-friendly.

That statement is not true

Why do you think that you need to copy libraries etc into the same folder as the .ino file before compiling ?

Sorry, maybe I stated my argument not that well. What I mean is that if U want to export your whole project to a folder that includes all the needed libraries you have to do it manually. For example if u want to send your uncompiled project to someone else who doesn't have the access to download or update libraries...

Updated the initial text a bit.....

You'd have to bundle the tool chain used as well if you are talking about archiving.

IDE version 1.x.x supports portable installation and would be a way to keep all of what went into building the code in one file.

I think, and I have seen it suggested. TBH I don't know where the compiler is.

FWIW I use several versions of the IDE, all are portable. I still keep my sketch folders outside the packages which are the portable installations.

a7

So I see
My answer was based on your original topic title. Now it is clearer what you want to do

Hi @Bertolli. You might be interested in the "build profiles" feature of Arduino CLI:

https://arduino.github.io/arduino-cli/latest/sketch-project-file/#build-profiles

Arduino CLI is the official command line tool provided by Arduino.

Arduino CLI will generate a build profile entry for all the dependencies currently in use if you add the --dump-profile flag to the arduino-cli compile command:

https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_compile/

In this way, you only need to have the sketch project file with that build profile and Arduino CLI will automatically install the exact library and boards platform dependencies of the sketch from the build profile data. This is especially useful for version control, where if you checked in the hundreds of megabytes of toolchain to the Git repository, it would cause the size of the repository to blow up to gigabytes over time.

Yes, I agree it was a bit unclear topic start at the beginning.

But anyway. It would be nice to have such function that all libraries and dependancies could be tied together and exported as one package.

What I also find problematic is the version management of the libraries; When some library.h is #included to the code it is difficult say from the .ino (code) itself what version of the library is used.

Sometimes newer versions of the libraries use different commands and if u want to build code based on code description that has only the name of the included library it doesn't tell what version was used. This can cause compiling errors because if u use the newest library it might not work the same way or with same commands as it was planned in the original (older) code?

Hi, have to check that.

Do you know in how long the library chains can generally be? I mean if the main cpp code uses library1.h and the

library1.h uses

library2.h and
library3.h to function...

Library2.h uses 5.h and 6.h
Library3.h uses 7.h and 8.h

It seems the chains can grow
very big (not that sure how this works)

Could it be possible to issue a rule or "trademark" of some sort of limited chain (PURE) libraries. The way they could work is that they are not allowed to include other libraries at all. This way coder knows that this library does not form any additionl chains and make the compiling so time consuming and sometimes a bit fragile?


I asked this question from the AI too :slight_smile:

Yes, it’s entirely possible to define and adopt a “PURE” library standard where libraries are restricted from including other libraries. This approach could indeed simplify dependency management, reduce compilation time, and make codebases more predictable and maintainable. Here’s how such a rule or trademark might work:

What Are PURE Libraries?

•	Definition: A PURE library is a self-contained library that does not include or depend on any other libraries.
•	Purpose: Ensures the library does not form additional chains, keeping dependency graphs flat and manageable.

Rules for PURE Libraries

1.	No External Includes:
•	The library must only use core C++/Arduino headers like <Arduino.h>, <stdint.h>, or standard C++ headers (e.g., <cmath>).
•	No includes of third-party libraries or other custom libraries.
2.	Self-Contained Code:
•	All necessary functionality must be implemented within the library itself or rely solely on Arduino core functionalities.
3.	No Implicit Dependencies:
•	The library cannot assume other libraries are included in the main sketch.
•	The library must operate independently.
4.	Minimal Functionality:
•	Each PURE library should serve a single, well-defined purpose (single-responsibility principle).

Benefits of PURE Libraries

1.	Faster Compilation:
•	Since there are no nested #include chains, the preprocessor has less work to do, and compilation times are significantly reduced.
2.	Improved Reliability:
•	PURE libraries are less fragile because they avoid dependency conflicts or version mismatches with other libraries.
3.	Simplified Debugging:
•	A flat dependency structure makes it easier to identify where errors originate.
4.	Predictable Behavior:
•	Users know exactly what functionality the library provides without worrying about hidden dependencies or side effects.
5.	Easier Integration:
•	PURE libraries are easier to integrate into larger projects because they don’t add additional dependencies.

Challenges of PURE Libraries

1.	Code Duplication:
•	Without the ability to include other libraries, common functionality might need to be re-implemented in multiple libraries, leading to redundancy.
2.	Limited Scope:
•	PURE libraries are best suited for small, modular tasks. Complex libraries (e.g., those managing hardware like sensors or displays) often rely on other libraries for low-level functionality.
3.	Lack of Extensibility:
•	Without including other libraries, PURE libraries may be harder to extend or adapt to new requirements.

How PURE Libraries Could Work in Practice

1.	PURE Certification/Trademark:
•	Libraries adhering to the PURE standard could be labeled with a “PURE Certified” badge or trademark.
•	This certification ensures developers know the library is self-contained and dependency-free.
2.	Naming Convention:
•	PURE libraries could follow a naming convention (e.g., PURE_Servo, PURE_Timer).
3.	Development Guidelines:
•	A formal set of rules and best practices could guide developers in creating PURE libraries.
4.	Tooling Support:
•	Arduino IDE or other development tools could include features to flag or validate PURE libraries.

Example of a PURE Library

PUREBlink.h:

#ifndef PUREBLINK_H
#define PUREBLINK_H

#include <Arduino.h> // Core Arduino functions only

class PUREBlink {
public:
PUREBlink(uint8_t pin);

void begin();       // Initialize the pin
void toggle();      // Toggle the pin state
void setInterval(unsigned long interval);

void update();      // Call in loop() to handle blinking

private:
uint8_t _pin;
unsigned long _interval;
unsigned long _lastToggle;
bool _state;
};

#endif

PUREBlink.cpp:

#include "PUREBlink.h"

PUREBlink::PUREBlink(uint8_t pin) : _pin(pin), _interval(1000), _lastToggle(0), _state(false) {}

void PUREBlink::begin() {
pinMode(_pin, OUTPUT);
}

void PUREBlink::toggle() {
_state = !_state;
digitalWrite(_pin, _state);
}

void PUREBlink::setInterval(unsigned long interval) {
_interval = interval;
}

void PUREBlink::update() {
if (millis() - _lastToggle >= _interval) {
toggle();
_lastToggle = millis();
}
}

Usage in Sketch:

#include <PUREBlink.h>

PUREBlink myBlink(13);

void setup() {
myBlink.begin();
myBlink.setInterval(500); // Blink every 500 ms
}

void loop() {
myBlink.update(); // Handles blinking
}

•	This PURE library provides basic blinking functionality without relying on any external dependencies.

Real-World Applications

1.	Small Embedded Systems:
•	On devices with limited resources, PURE libraries ensure minimal memory usage and fast compilation.
2.	Educational Projects:
•	Students and beginners can use PURE libraries without worrying about complex dependency chains.
3.	Certification for Industrial Use:
•	In industrial applications, ensuring the reliability of libraries is crucial. PURE libraries provide predictable behavior.

Conclusion

PURE libraries offer a promising approach to simplify Arduino development by limiting dependency chains. While they may not be suitable for every scenario, they provide significant benefits in terms of reliability, maintainability, and performance. By establishing clear rules and promoting a certification standard, developers can create and share self-contained libraries that streamline development and reduce the challenges of dependency management.

There is no limit.

Arduino CLI does not have any such capability and such a thing is out of scope for the Arduino CLI project.

If this is something that would be useful to you, I suggest you look around to see if there is an existing tool for C++. It is possible that could be used with Arduino projects as well.

Although determining the dependencies chain might be difficult, determining the number of library dependencies is simple. So if you don't find an existing tool that provides the exact functionality you hoped for, you could consider implementing your own system that simply puts a limit on the number of total library dependencies of a project. If you add the --json flag to the arduino-cli compile command, it will produce machine readable output. The builder_result.used_libraries property of this output is an array of the library dependencies, so you only need to create a system which parses the output and determines the length of that array.

There are many excellent tools and libraries for parsing the JSON data format of this Arduino CLI output. One such tool is jq. Here is an example of determining the number of library dependencies of the sketch using jq:

$ arduino-cli compile --fqbn arduino:avr:uno --json | jq '.builder_result.used_libraries | length'
3

This is the sketch that was compiled:

#include <Servo.h>
#include <Ethernet.h>
void setup() {}
void loop() {}

Even though the sketch only contains #include directives for two libraries, the "Ethernet" library has a dependency on the SPI library, and thus the sketch has three library dependencies.

If you add the --only-compilation-database flag to the command, compilation will be skipped and Arduino CLI will only generate the list of libraries, which makes the process more efficient:

arduino-cli.exe compile --fqbn arduino:avr:uno --json --only-compilation-database | jq '.builder_result.used_libraries | length'

Of course this simple command only determined the number of dependencies. Some additional code would be needed to check whether the number exceeded the arbitrary limit you have imposed on the project. But that is easily done.

So every library must implement all the capabilities it requires, even though such capabilities are already provided by another library. I think the drawbacks of this would likely outweigh the benefits. Many Arduino libraries accept arbitrary objects that implement a standardized API such as the Stream, Print, or Client class. This means we can use those libraries in combination with other libraries that the library developer doesn't even know existed (and may not have even existed at the time they created the library).

And there is the other problem that there is no way you can impose the requirement of creating "PURE" libraries on the vast community of Arduino library developers so the only way you could accomplish this is by converting each library you want to use to "PURE" and then taking on the responsibility for maintaining that "PURE" version of the library from there on.

It is true that there is a problem where we might break a project by updating a transitive dependency to a version that is incompatible with the dependent library, but the Arduino libraries system allows the library developer to specify the dependencies of their library, including version constraints:

https://arduino.github.io/arduino-cli/latest/library-specification/#version-constraints

That, in combination with build profiles solves this problem, and does so in a manner that is easily accomplished (as long as you don't mind using command line tools).

Ok thanks for the answer. I was also thinking that one solution could be such that you could combine libraries.

If you have libray that includes 2-3 sub libraries. You could have a button that combines multiple libraries.h into one "biglibrary.h". The
combined library would include all the necessary code lines from the sub libraries. This way the sub dependencies would disappear from the initial library.h that is used in the arduino project?

For example if you have library called color.h that includes libraries red.h, green.h and blue.h. You could press combine color.h which combines all these to one library called colorCOMB.h

You would never want to merge the files. The only reasonable approach would be to simply bundle all the files of the dependency with the library. The term for this is "vendoring".

Auto vendoring function could be handy?

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.