4
submitted 11 months ago by mikaey00@alien.top to c/homelab@selfhosted.forum

So I took it upon myself to answer the question of “what are the best and cheapest microSD cards out there?” This includes evaluating whether they’re fake flash, how well they perform, and how many read/write cycles they can endure before they start failing. So far I’ve tested four to the point of failure, I have 37 being tested right now, and I have 21 more waiting to be tested.

Sorry for the horrendous cable management — I have cable ties on order.

top 35 comments
sorted by: hot top controversial new old
[-] Pvt-Snafu@alien.top 2 points 11 months ago

That actually looks cool. Are you looking to make sort of an article or blog post and publish the results?

[-] mikaey00@alien.top 2 points 11 months ago

I'm actually working on a script for a YouTube video right now. Might also make a blog post about it, I dunno.

[-] Amabry@alien.top 1 points 11 months ago

"Stress testing micro SD cards" is just a funny way of saying "Prematurely burning up a bunch of SD cards.

[-] Professional-Fee2235@alien.top 1 points 11 months ago

Well, yeah but you want to know which one burns the slowest

[-] 1980sumthing@alien.top 1 points 11 months ago

What is the combined max writing speed to all the cards, does the usb max out? At what numbers? And what is the model of the hub?

[-] comfydecal@infosec.pub 1 points 11 months ago

Can you post your results when you find the best?

[-] Friendly_Engineer_@alien.top 1 points 11 months ago

FYI, OP doesn’t care about the data or whatnot. They just really enjoy torturing hardware.

[-] mikaey00@alien.top 1 points 11 months ago

*taps nose*

[-] ita5248@alien.top 1 points 11 months ago

Can you recommend reliable USB hub with at least 10 ports?

[-] pjockey@alien.top 1 points 11 months ago

Why cable manage a temporary setup (especially with zip ties)?

[-] Real_MakinThings@alien.top 1 points 11 months ago

Have fun. Where I see this beating out other options for low access / possibly cold storage is you could make it incredibly dense in terms of tb/kg and tb/cm3. It would make a fun challenge and be damn near impossible to maintain 🤣

[-] Professional-Fee2235@alien.top 1 points 11 months ago

Flash is... not great for cold storage from what I've read. Electrons eventually leak out and your data is gone. Well, unless you put it in freezer

Fun fact: JEDEC specification for commerical flash only specifies year of retention as minimum at 30C and much lower at elevated temperature

https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/tp618-ssd-tech-paper-us.pdf

[-] dnabre@alien.top 1 points 11 months ago

As co-workers have reminded me many times, 127 device per bus limit is not intended as a challenge.

[-] Few_Flamingo_7716@alien.top 1 points 11 months ago

You are doing gods work, salute!🫡

[-] Xkaper@alien.top 1 points 11 months ago

Raid 0 that mess just for fun lad.

[-] drfusterenstein@alien.top 1 points 11 months ago

Reminds me of the u/spaceinvaderone video where he stress tests a load of usb sticks.

[-] xXx_HardwareSwap_Alt@alien.top 1 points 11 months ago

Let me know which is the best so I can buy it for my raspberry pi.

[-] Professional-Fee2235@alien.top 1 points 11 months ago

Which header you're using? I had bad luck with readers being pretty wonky.

Would be interested in results, my SBC cluster died on a bunch of samsung EVO cards turning read-only.

Or rather PRETENDING to be read-write by ignoring writes, which made very funny debugging session where the OS started crashing the moment write cache in RAM ran out and it tried to read the data it wrote...

[-] JVAV00@alien.top 1 points 11 months ago

Are Linus tech tips

[-] bicebird@alien.top 1 points 11 months ago

Not sure SD's are competitive for performance / durability but it'd be interesting to see (presumably) how much lower the power consumption is, especially if you can disable / enable the usb port they're connected to on demand for something like media storage where an extra second or so startup time isn't an issue,

[-] ChumpyCarvings@alien.top 1 points 11 months ago

I use h2testw, did you find any outright fakes (died first pass?)

[-] definitlyitsbutter@alien.top 1 points 11 months ago

Boi that is great! Do you publish your data/experiences somwhere? And also did you throw some of the high/ultra endurance models in the mix (normally for video surveillance and stuff)? I had so many normal micro SDs fail for i dont trust them anymore and rather grab one of these....

[-] alias4007@alien.top 1 points 11 months ago

Do you have any temperature controls for this experiment? Seems that all that gear is heating the sdcards under test. It would be interesting to see their performance under temperature extremes, or at least under typical ambient operating temperatures.

[-] RedditNotFreeSpeech@alien.top 1 points 11 months ago

If dashcams are any indication, they are mostly shit. Even the endurance cards don't last as long as you'd expect.

[-] Fast-String486@alien.top 1 points 11 months ago

I would put a fan on those card readers though to remove them as a point of failure👀... I've needed to backup 7 micro sd cards at a time for some shoots using a USB hub and damm do the readers get hot when copying a ton of stuff

[-] jfoster0818@alien.top 1 points 11 months ago

I also have a boatload of these adapters and no practical use lol

[-] Vinez_Initez@alien.top 1 points 11 months ago

You are not stress testing anything here besides the Universal Serial Bus...

[-] SherSlick@alien.top 1 points 11 months ago

The point seems to be a write endurance test mostly

[-] Antrikshy@alien.top 1 points 11 months ago

If you read the post, they're testing for endurance, not speed.

[-] AJsRaceway@sh.itjust.works 1 points 11 months ago

That's awesome. I can't wait to see the results. That's quite a collection of card readers, also. It will be interesting to see if they all survive, too.

[-] Buttafuoco@alien.top 0 points 11 months ago

What are your methods for testing?

[-] mikaey00@alien.top 2 points 11 months ago

So I'm testing for three things:

  • Capacity
  • Performance
  • Endurance

Capacity

Divide the card up into eight equal segments. Within each segment (except for the first), choose a random starting location. Write 4MB of random data to the very beginning of the card, the very end of the card, and to each of the other 7 randomly selected locations. Then, read back each segment and compare the data read to what was written. If all 9 segments match, the card is considered "not fake flash". Otherwise, the space between the last "good" segment and the first "bad" segment is bisected until the point at which "bad" sectors begin to appear is determined. The device is considered to be "fake flash"; the capacity is declared to be the amount of space between the start of the device and the last "good" sector.

It's not a perfect strategy -- cause it doesn't account for wraparound flash or flash that is bad from the very beginning -- but it gets it right 98% of the time.

Performance

There's two components to this: a sequential read/write test and a random read/write test.

For the sequential read/write test: start from the very beginning of the device. For 30 seconds, read as much data as possible in a sequential fashion, recording the total number of bytes read. At the end of 30 seconds, take the total number of bytes read and divide by 30. The device's sequential read speed is considered to be this result (measured in bytes per second). For the sequential write test -- do the same thing, but generate random data and write it to the device.

For the random read/write test: Choose a random sector on the device. Read 4KB of data from the chosen sector. Repeat this process for 30 seconds, keeping count of the number of read operations completed. Take the final result and divide by 30. The device's random read speed is considered to be this result (measured in IOPs per second). For the random write test -- do the same thing, but generate random data and write it to the device.

Again, this isn't perfect. It's probably good enough for determining whether a device meets the requirements for Class2/Class4/Class6/Class10, but the video speed classes (V6/V10/V30/V60/V90) require some special commands to be issued to the card that just can't be issued via most USB card readers. Also, I didn't read the specific set of conditions for testing for the A1/A2 speed classes -- so my program can probably get you in the neighborhood of the right number, but won't be 100% accurate.

Endurance

Divide the card into 16 equal segments. Shuffle the segments into a random order. For each segment, write pseudorandom data to the entire segment.

Shuffle the segments into a random order again. For each segment, read back the data. Regenerate the pseudorandom data that was originally written to the segment and compare it to what was written. If there is a mismatch in any given sector, the sector is flagged as "bad".

(If an I/O error is encountered on any given sector, the operation is retried several times -- including resetting the device if necessary and/or waiting for the device to be reconnected. If all attempts fail, the sector is flagged as "bad".)

Keep a count of the number of complete read/write cycles that have been performed against the device.

Repeat until at least 50% of the sectors on the device have been flagged as "bad", or until an unrecoverable I/O error has occurred.

[-] Buttafuoco@alien.top 1 points 11 months ago

If the data being written is random how do you know what was written? Is it a known pattern? Something calculated given its address space?

For perf randread/write test, why was 4KB chosen?

For endurance do you have access to any hardware metrics to compare the running test? The drives should be self reporting endurance, each vendor may vary slightly in how they express it

[-] mikaey00@alien.top 1 points 11 months ago

If the data being written is random how do you know what was written? Is it a known pattern?

Essentially, yes. I use the random()/srandom() calls in the C library, which generates random numbers in a deterministic fashion using a given seed as the starting point. I generate unique seeds for each segment and each pass. When reading back a particular segment, I just set the seed back to the same value that I used when I originally wrote the data.

For perf randread/write test, why was 4KB chosen?

Because that's what's dictated by the SD card specification.

For endurance do you have access to any hardware metrics to compare the running test? The drives should be self reporting endurance, each vendor may vary slightly in how they express it

Nope. SD cards don't have SMART data like hard drives/SSDs do. I can't find anything in the SD spec that tells you how to figure out how long the card has been operating or how much data has been written to it.

[-] Buttafuoco@alien.top 1 points 11 months ago

Thanks for these thorough responses 😃.

this post was submitted on 21 Nov 2023
4 points (100.0% liked)

Homelab

371 readers
2 users here now

Rules

founded 1 year ago
MODERATORS