mcndave/spout
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update FAQ

Browse Source
This commit is contained in:
Adrien Loison 2019-05-24 09:16:59 +02:00 committed by Adrien Loison
parent 3beaa32021
commit 9d33fcdd00
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/_pages/faq.md
View File

@ -4,15 +4,15 @@ title: Frequently Asked Questions
permalink: /faq/ permalink: /faq/
--- ---
### How can Spout handle such large data sets and still use less than 3MB of memory? ### How can {{ site.spout_html }} handle such large data sets and still use less than 3MB of memory?
When writing data, Spout is streaming the data to files, one or few lines at a time. That means that it only keeps in memory the few rows that it needs to write. Once written, the memory is freed. When writing data, {{ site.spout_html }} is streaming the data to files, one or few lines at a time. That means that it only keeps in memory the few rows that it needs to write. Once written, the memory is freed.
Same goes with reading. Only one row at a time is stored in memory. A special technique is used to handle shared strings in XLSX, storing them - if needed - into several small temporary files that allows fast access. Same goes with reading. Only one row at a time is stored in memory. A special technique is used to handle shared strings in XLSX, storing them - if needed - into several small temporary files that allows fast access.
### How long does it take to generate a file with X rows? ### How long does it take to generate a file with X rows?
Here are a few numbers regarding the performance of Spout: Here are a few numbers regarding the performance of {{ site.spout_html }}:
| Type | Action | 2,000 rows (6,000 cells) | 200,000 rows (600,000 cells) | 2,000,000 rows (6,000,000 cells) | | Type | Action | 2,000 rows (6,000 cells) | 200,000 rows (600,000 cells) | 2,000,000 rows (6,000,000 cells) |
|------|-------------------------------|--------------------------|------------------------------|----------------------------------| |------|-------------------------------|--------------------------|------------------------------|----------------------------------|
@ -24,7 +24,7 @@ Here are a few numbers regarding the performance of Spout:
| ODS | Read | 1 second | 1-2 minutes | 5-6 minutes | | ODS | Read | 1 second | 1-2 minutes | 5-6 minutes |
| | Write | < 1 second | 35-40 seconds | 5-6 minutes | | | Write | < 1 second | 35-40 seconds | 5-6 minutes |
### Does Spout support charts or formulas? ### Does {{ site.spout_html }} support charts or formulas?
No. This is a compromise to keep memory usage low. Charts and formulas requires data to be kept in memory in order to be used. No. This is a compromise to keep memory usage low. Charts and formulas requires data to be kept in memory in order to be used.
So the larger the file would be, the more memory would be consumed, preventing your code to scale well. So the larger the file would be, the more memory would be consumed, preventing your code to scale well.