Windows 11 does use more RAM than Windows 10. If you look at the minimum system requirements, you can see that Windows 10 (64-bit) requires at least 2 GB of RAM and Windows 11 requires at least 4 GB of RAM.
When the capacity of available RAM is being fully used, then Windows starts to use a disk as RAM (so called virtual memory). Using of disk as RAM is slowing down the computer, because disk can't read or write data as fast as RAM.
Both Windows 10 and Windows 11 use quite a lot of RAM even on idle, so it is recommended to have at least 8 GB of RAM.
According to my testings, the average usage of RAM is:
Windows 10 (64-bit)
2.6 GB - 3.1 GB on idle, after windows startup
4.0 GB - 4.5 GB when browsing the web and watching youtube video
Windows 11
3.2 GB - 3.8 GB on idle, after windows startup
4.8 GB - 5.2 GB when browsing the web and watching youtube video