Memory Management #2

New issue

Open

opened 2026-01-18 20:08:07 -05:00 by Ghost · 1 comment

Ghost commented 2026-01-18 20:08:07 -05:00 (Migrated from codefloe.com)

Copy link

Simply put, the memory management is shit, like, really shit. For example, the MD5 hash of a 35M file takes 1.79s in nclang, while using md5sum it takes 0.040s. This issue will serve as a tracker for attempts and other activities related to improving this.

❯ strace -c ./md5 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
052dfd1c4c094ec292494c7ef7aef279  ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 68.12    0.002515           0      8504           read
 25.38    0.000937           0      2837           brk
  2.22    0.000082          10         8           mmap
  0.70    0.000026           8         3           mprotect
  0.60    0.000022           4         5           rt_sigaction
  0.43    0.000016           8         2           openat
  0.35    0.000013          13         1           munmap
  0.30    0.000011           5         2           rt_sigprocmask
  0.27    0.000010          10         1           open
  0.24    0.000009           4         2           fstat
  0.24    0.000009           4         2           pread64
  0.22    0.000008           2         3           close
  0.19    0.000007           7         1         1 access
  0.14    0.000005           5         1           arch_prctl
  0.14    0.000005           5         1           getrandom
  0.14    0.000005           5         1           rseq
  0.11    0.000004           4         1           set_tid_address
  0.11    0.000004           4         1           set_robust_list
  0.11    0.000004           4         1           prlimit64
  0.00    0.000000           0         2           write
  0.00    0.000000           0         1           execve
------ ----------- ----------- --------- --------- ----------------
100.00    0.003692           0     11380         1 total
❯ strace -c md5sum ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
052dfd1c4c094ec292494c7ef7aef279  ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 97.43    0.000987           0      1066           read
  1.09    0.000011          11         1           write
  1.09    0.000011           1         7           close
  0.39    0.000004           0         6           fstat
  0.00    0.000000           0        14           mmap
  0.00    0.000000           0         4           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         2           pread64
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         1           fadvise64
  0.00    0.000000           0         5           openat
  0.00    0.000000           0         1           set_robust_list
  0.00    0.000000           0         1           prlimit64
  0.00    0.000000           0         1           getrandom
  0.00    0.000000           0         1           rseq
------ ----------- ----------- --------- --------- ----------------
100.00    0.001013           0      1118         1 total

Simply put, the memory management is shit, like, really shit. For example, the MD5 hash of a 35M file takes `1.79s` in nclang, while using `md5sum` it takes `0.040s`. This issue will serve as a tracker for attempts and other activities related to improving this. ```sh ❯ strace -c ./md5 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe 052dfd1c4c094ec292494c7ef7aef279 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 68.12 0.002515 0 8504 read 25.38 0.000937 0 2837 brk 2.22 0.000082 10 8 mmap 0.70 0.000026 8 3 mprotect 0.60 0.000022 4 5 rt_sigaction 0.43 0.000016 8 2 openat 0.35 0.000013 13 1 munmap 0.30 0.000011 5 2 rt_sigprocmask 0.27 0.000010 10 1 open 0.24 0.000009 4 2 fstat 0.24 0.000009 4 2 pread64 0.22 0.000008 2 3 close 0.19 0.000007 7 1 1 access 0.14 0.000005 5 1 arch_prctl 0.14 0.000005 5 1 getrandom 0.14 0.000005 5 1 rseq 0.11 0.000004 4 1 set_tid_address 0.11 0.000004 4 1 set_robust_list 0.11 0.000004 4 1 prlimit64 0.00 0.000000 0 2 write 0.00 0.000000 0 1 execve ------ ----------- ----------- --------- --------- ---------------- 100.00 0.003692 0 11380 1 total ❯ strace -c md5sum ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe 052dfd1c4c094ec292494c7ef7aef279 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 97.43 0.000987 0 1066 read 1.09 0.000011 11 1 write 1.09 0.000011 1 7 close 0.39 0.000004 0 6 fstat 0.00 0.000000 0 14 mmap 0.00 0.000000 0 4 mprotect 0.00 0.000000 0 1 munmap 0.00 0.000000 0 3 brk 0.00 0.000000 0 2 pread64 0.00 0.000000 0 1 1 access 0.00 0.000000 0 1 execve 0.00 0.000000 0 1 arch_prctl 0.00 0.000000 0 1 set_tid_address 0.00 0.000000 0 1 fadvise64 0.00 0.000000 0 5 openat 0.00 0.000000 0 1 set_robust_list 0.00 0.000000 0 1 prlimit64 0.00 0.000000 0 1 getrandom 0.00 0.000000 0 1 rseq ------ ----------- ----------- --------- --------- ---------------- 100.00 0.001013 0 1118 1 total ```

Ghost commented 2026-03-22 13:26:03 -04:00 (Migrated from codefloe.com)

Copy link

❯ strace -c ./md5 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
052dfd1c4c094ec292494c7ef7aef279  ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 90.26    0.000352           0      1065           read
  2.31    0.000009           9         1           open
  2.31    0.000009           1         5           rt_sigaction
  2.05    0.000008           4         2           rt_sigprocmask
  1.79    0.000007           2         3           brk
  1.28    0.000005           0         9           mmap
  0.00    0.000000           0         2           write
  0.00    0.000000           0         3           close
  0.00    0.000000           0         2           fstat
  0.00    0.000000           0         3           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         2           pread64
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         2           openat
  0.00    0.000000           0         1           set_robust_list
  0.00    0.000000           0         1           prlimit64
  0.00    0.000000           0         1           getrandom
  0.00    0.000000           0         1           rseq
------ ----------- ----------- --------- --------- ----------------
100.00    0.000390           0      1108         1 total
❯ strace -c md5sum ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
052dfd1c4c094ec292494c7ef7aef279  ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 48.63    0.000338           0      1066           read
 20.00    0.000139         139         1           execve
 12.37    0.000086           6        14           mmap
  4.32    0.000030           7         4           mprotect
  3.88    0.000027           5         5           openat
  2.01    0.000014           2         6           fstat
  1.73    0.000012           1         7           close
  1.44    0.000010           3         3           brk
  1.29    0.000009           9         1           munmap
  0.86    0.000006           3         2           pread64
  0.58    0.000004           4         1         1 access
  0.43    0.000003           3         1           arch_prctl
  0.43    0.000003           3         1           fadvise64
  0.43    0.000003           3         1           set_robust_list
  0.43    0.000003           3         1           prlimit64
  0.43    0.000003           3         1           getrandom
  0.43    0.000003           3         1           rseq
  0.29    0.000002           2         1           set_tid_address
  0.00    0.000000           0         1           write
------ ----------- ----------- --------- --------- ----------------
100.00    0.000695           0      1118         1 total

```sh ❯ strace -c ./md5 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe 052dfd1c4c094ec292494c7ef7aef279 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 90.26 0.000352 0 1065 read 2.31 0.000009 9 1 open 2.31 0.000009 1 5 rt_sigaction 2.05 0.000008 4 2 rt_sigprocmask 1.79 0.000007 2 3 brk 1.28 0.000005 0 9 mmap 0.00 0.000000 0 2 write 0.00 0.000000 0 3 close 0.00 0.000000 0 2 fstat 0.00 0.000000 0 3 mprotect 0.00 0.000000 0 1 munmap 0.00 0.000000 0 2 pread64 0.00 0.000000 0 1 1 access 0.00 0.000000 0 1 execve 0.00 0.000000 0 1 arch_prctl 0.00 0.000000 0 1 set_tid_address 0.00 0.000000 0 2 openat 0.00 0.000000 0 1 set_robust_list 0.00 0.000000 0 1 prlimit64 0.00 0.000000 0 1 getrandom 0.00 0.000000 0 1 rseq ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000390 0 1108 1 total ❯ strace -c md5sum ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe 052dfd1c4c094ec292494c7ef7aef279 ../../Downloads/DCS_MovingMap/DCS_MovingMap.exe % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 48.63 0.000338 0 1066 read 20.00 0.000139 139 1 execve 12.37 0.000086 6 14 mmap 4.32 0.000030 7 4 mprotect 3.88 0.000027 5 5 openat 2.01 0.000014 2 6 fstat 1.73 0.000012 1 7 close 1.44 0.000010 3 3 brk 1.29 0.000009 9 1 munmap 0.86 0.000006 3 2 pread64 0.58 0.000004 4 1 1 access 0.43 0.000003 3 1 arch_prctl 0.43 0.000003 3 1 fadvise64 0.43 0.000003 3 1 set_robust_list 0.43 0.000003 3 1 prlimit64 0.43 0.000003 3 1 getrandom 0.43 0.000003 3 1 rseq 0.29 0.000002 2 1 set_tid_address 0.00 0.000000 0 1 write ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000695 0 1118 1 total ```

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

No results found.

Labels

Clear labels   

Compat/Breaking

Breaking change that won't be backward compatible

  

Kind/Bug

Something is not working

  

Kind/Documentation

Documentation changes

  

Kind/Enhancement

Improve existing functionality

  

Kind/Feature

New functionality

  

Kind/Security

This is security issue

  

Kind/Testing

Issue or pull request related to testing

  

Priority/Critical

The priority is critical

  

Priority/High

The priority is high

  

Priority/Low

The priority is low

  

Priority/Medium

The priority is medium

  

Reviewed/Confirmed

Issue has been confirmed

  

Reviewed/Duplicate

This issue or pull request already exists

  

Reviewed/Invalid

Invalid issue

  

Reviewed/Won't Fix

This issue won't be fixed

  

Status/Abandoned

Somebody has started to work on this but abandoned work

  

Status/Blocked

Something is blocking this issue or pull request

  

Status/Need More Info

Feedback is required to reproduce issue or to continue work

No labels

Compat/Breaking

Kind/Bug

Kind/Documentation

Kind/Enhancement

Kind/Feature

Kind/Security

Kind/Testing

Priority/Critical

Priority/High

Priority/Low

Priority/Medium

Reviewed/Confirmed

Reviewed/Duplicate

Reviewed/Invalid

Reviewed/Won't Fix

Status/Abandoned

Status/Blocked

Status/Need More Info

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

nclang/ncc#2

No description provided.