Skip to content

Commit 606b9ea

Browse files
dedece35dedece35
authored
Merge branch 'main' into OptimizeSquareComputation
2 parents b1b801d + d795d41 commit 606b9ea

5 files changed

Lines changed: 94 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1010
### Added
1111

1212
- Add specifications for rule GCI99, this rule concerns the different ways of squaring a value.
13+
- [#382](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/382) Add specifications for rule GCI96, this rule is specific to Python because it's based on the `pandas` library, a library used for data.
1314
- [#400](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/400) Add rule GCI535 - Prefer usage of Intl.NumberFormat
1415

1516
### Changed

‎RULES.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ Some are applicable for different technologies.
7373
| GCI94 | Use orElseGet instead of orElse | Parameter of orElse() is evaluated, even when having a non-empty Optional. Supplier method of orElseGet passed as an argument is only executed when an Optional value isn’t present. Therefore, using orElseGet() will save computing time. | [Optimized use of Java Optional Else](https://github.com/green-code-initiative/creedengo-challenge/issues/77) | ✅ | 🚫 | 🚫 | 🚫 | 🚫 | 🚫 | 🚫 |
7474
| GCI95 | Avoid querying SQL columns that are not used | When a database is queried without an ORM, every column that is queried should be used afterward somewhere in the function. | [cnumr best practices (3rd edition) BP_075](https://github.com/cnumr/best-practices/blob/main/chapters/BP_075_fr.md) | 🚧 | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ |
7575
| GCI99 | OptimizeSquareComputation | In Python, if you want to square x, you can do x*x or x**2. | | ❓ | ❓ | ❓ | ✅ | ❓ | ❓ | ❓ |
76+
| GCI96 | DATA/AI Pandas - Avoid Reading Unnecessary Columns in CSV Files | Reading CSV files without explicitly specifying which columns to load leads to unnecessary data loading and increases memory and energy consumption. | | 🚫 | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 |
7677
| GCI203 | Detect unoptimized file formats | When it is possible, to use svg format image over other image format | | 🚧 | 🚀 | 🚀 | ✅ | 🚀 | 🚀 | 🚫 |
7778
| GCI404 | Avoid list comprehension in iterations | Use generator comprehension instead of list comprehension in for loop declaration | | 🚫 | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 |
7879
| GCI522 | Sobriety: Brightness Override | To avoid draining the battery, iOS and Android devices adapt the brightness of the screen depending on the environment light. | | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 | 🚫 |

‎src/main/rules/GCI96/GCI96.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"title": "DATA/AI Pandas - Avoid Reading Unnecessary Columns in CSV Files",
3+
"type": "CODE_SMELL",
4+
"status": "ready",
5+
"remediation": {
6+
"func": "Constant\/Issue",
7+
"constantCost": "10min"
8+
},
9+
"tags": [
10+
"creedengo",
11+
"eco-design",
12+
"performance",
13+
"data",
14+
"ai",
15+
"pandas"
16+
],
17+
"defaultSeverity": "Minor"
18+
}

‎src/main/rules/GCI96/python/GCI96.asciidoc‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
This rule is specific to Python because it's related to the Pandas library, which is widely used for data manipulation and analysis in Python.
2+
3+
Reading CSV files without explicitly specifying which columns to load leads to unnecessary data loading and increases memory and energy consumption. This guidance is specific to the use of the Pandas library in Python, but it aligns with the more general GCI74: Avoid SELECT * from table in SQL. To ensure low environmental impact and optimal performance, always use the usecols parameter in pandas.read_csv() to select only the required columns.
4+
5+
== Non Compliant Code Example
6+
7+
[source,python]
8+
----
9+
file_path = 'data.csv'
10+
df = pd.read_csv(file_path)
11+
# or
12+
df = pd.read_csv('data.csv')
13+
----
14+
15+
In this case, **all columns** are read into memory, even if only one or two are needed.
16+
17+
== Compliant Solution
18+
19+
[source,python]
20+
----
21+
file_path = 'data.csv'
22+
df = pd.read_csv(file_path, usecols=['A', 'B']) # Only read needed columns
23+
----
24+
25+
This ensures only the necessary data is loaded, reducing memory usage and energy consumption.
26+
27+
== Relevance Analysis
28+
29+
Local experiments were conducted to assess the environmental impact of reading CSV files with and without column selection.
30+
31+
=== Configuration
32+
33+
* Processor: Intel(R) Core(TM) Ultra 5 135U, 2100 MHz, 12 cores, 14 logical processors
34+
* RAM: 16 GB
35+
* CO₂ Emissions Measurement: https://mlco2.github.io/codecarbon/[CodeCarbon]
36+
37+
=== Context
38+
39+
We generated CSV files with the following row sizes:
40+
* 1,000
41+
* 10,000
42+
* 100,000
43+
* 1,000,000
44+
45+
Each file contains 5 columns (`A`, `B`, `C`, `D`, `E`). We measured the carbon emissions required to read:
46+
* 1 column
47+
* 2 columns
48+
* 3 columns
49+
* 4 columns
50+
* all 5 columns
51+
52+
=== Impact Analysis
53+
54+
The graph below illustrates the emissions generated as a function of file size and number of columns read.
55+
56+
*Carbon emissions during reading (kgCO₂eq):*
57+
58+
image::image.png[]
59+
60+
The results show a **increase in emissions** as more columns are read, and a strong correlation between file size and emissions.
61+
62+
== Conclusion
63+
64+
The rule is relevant. Explicitly specifying columns:
65+
- Reduces carbon emissions
66+
- Decreases memory and CPU usage
67+
- Improves data loading time
68+
69+
This is especially critical when working with large datasets or in environments where sustainability and performance matter (e.g., cloud computing, machine learning pipelines).
70+
71+
== References
72+
https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html
73+
https://medium.com/@amit25173/what-is-usecols-in-pandas-7a6a43885f4b
74+

‎src/main/rules/GCI96/python/image.png‎

78.1 KB
Loading

0 commit comments

Comments
 (0)