Hi there, I am trying to setup a high-availability high-performance Storage and I have no idea where to start at all. and I have a list of question i hope you could help me with.
Do i need SAN or DAS that connect to multiple control node?
What file system? and I want to avoid Lusterfs
What hardware do i need?
What Os do i use? i ideally want to use truenas but i am comfortable with another distro.
the storage need to beable to scale to petabyte level and ideally FAST
You can go with a single large capacity server, such as: https://www.45drives.com/products/storinator-xl60-configurations.php Or go with a cluster and a distributed file system. It's a lot less work/less complexity to go with a large single server. Performance wise, you either care most about the cost and use hard drives or you care most about speed and use some kind of solid state storage.
If I were you, I'd call a few storage vendors and see what they say. Compare feature vs cost and then decide if you want to roll your own.