Object Storage

• MinIO vs Amazon S3, as object storage system:
  • cost:
    • MinIO has two plans, both starting at $24,000 per year.
      S3 pricing is tiered, with the first 50 TB per month costing $0.023 per GB per month.
  • performance: …
• Ref: https://aws.amazon.com/s3/?nc1=h_ls
  • Amazon S3: Simple Storage Service
  • Object storage service, built to retrieve any amount of data from anywhere
  • S3's free tier offers a 12 month free trial that provides 5GB of Amazon S3 storage in the S3 Standard storage class;
    20,000 GET Requests; 2,000 PUT, COPY, POST, or LIST Requests; and 100 GB of Data Transfer Out each month.
    To learn more about S3's free tier offering and cost effective pricing options, visit the Amazon S3 pricing page.
  • Use case:
    • Build a data lake:
      • a centralized repository that allows you to store all your structured and unstructured data at any scale.
    • to run data analytics, artificial intelligence (AI), machine learning (ML), and high-performance computing (HPC) applications to unlock the value of your data
• Ref: https://min.io/
  • The World's Fastest Growing Object Store
  • AiStor: Hyperscale Object Store for AI
  • Customers who repatriate from the public cloud save 60% or more on average.
  • AIStor is Native to the AI Ecosystem
    AI revolution is powered by technologies that are either built on MinIO or integrated with it.
  • AIStor supports the full catalog of object-based protocols and API
    • S3
    • FTP & SFTP protocols
    • S3 over RDMA
    • AIHub
      • A private repository for storing AI models and datasets directly on AIStor.
        AIHub allows enterprises to create their own data and model repositories on the private cloud or in air-gapped environments without changing a single line of code.
    • promptObject
      • This powerful addition to the S3 API enables users to “talk” to objects like they would an LLM.
  • Ref: https://www.ibm.com/docs/SSBS6K_3.1.2/manage_cluster/minio.html#:~:text=Minio%20is%20a%20high%2Dperformance,using%20Amazon%20S3%20REST%20APIs.
  • Minio is a high-performance distributed Object Storage server, which is designed for large-scale private cloud infrastructure. Minio aggregates persistent volumes (PVs) into scalable distributed Object Storage, by using Amazon S3 REST APIs.
  • disadvantage of minIO
    • Ref: https://www.peerspot.com/products/minio-pros-and-cons#:~:text=It%20doesn't%20allow%20storage,for%20users%20seeking%20expandable%20solutions.
    • It doesn't allow storage expansion, requiring a multi-tenant approach. MinIO's free version limits storage, which can be restrictive for users seeking expandable solutions.
• Generative AI:
  • Cost
    MinIO has two plans, both starting at $24,000 per year. S3 pricing is tiered, with the first 50 TB per month costing $0.023 per GB per month.
  • Performance
    In a comparison of MinIO and Amazon S3 for Apache Spark workloads, MinIO outperformed Amazon S3 in most queries. MinIO is also strictly consistent, while Amazon S3 is only eventually consistent.
  • Use
    MinIO is designed for large-scale, multi-data center cloud storage services. It's hosted locally and implements the same public API as Amazon S3, so applications that work with Amazon S3 can also work with MinIO.
  • Storage classes
    S3 Glacier is for archival storage, with variable retrieval rates ranging from minutes to hours. S3 Glacier Deep Archive is for data that's only accessed once or twice per year.
  • Compatibility
    MinIO's support of S3 allows applications to work with data regardless of where it's located. This can help enterprises save money while optimizing for other factors.
  • Reference:
    https://blog.min.io/benchmarking-apache-spark-vs-aws-s3/
    https://min.io/product/s3-compatibility#:~:text=The%20Cost%20of%20S3%20Compatible,security%20and%20zero%20lock%2Din.
    https://www.univention.com/blog-en/2018/03/minio-the-secure-alternative-to-amazon-s3-for-object-storage/#:~:text=So%2C%20What%20Is%20Minio?,address%20of%20the%20Minio%20instance.
    https://min.io/product/overview#:~:text=MinIO%20is%20designed%20for%20large%20scale%2C%20multi%2Ddata,any%20disruption%20on%20upgrade%2C%20update%2C%20security%20incidents.
    https://bluexp.netapp.com/blog/aws-cvo-blg-s3-storage-the-complete-guide
    https://www.saasworthy.com/product/min-io/pricing#:~:text=MinIO%20FAQs%20The%20pricing%20for%20MinIO%20starts,year.%20MinIo%20Enterprise%20at%20$24000.00%20per%20year.

    The Ultimate Guide to AWS S3 Pricing

    
    https://min.io/resources/docs/MinIO-Throughput-Benchmarks-on-HDD-24-Node.pdf#:~:text=The%20benchmark%20results%20demonstrate%20that%20MinIO%20is,from%20a%2016%20node%20MinIO%20Object%20Server.
    https://repost.aws/knowledge-center/s3-maximum-transfer-speed-ec2#:~:text=Resolution.%20Traffic%20between%20Amazon%20EC2%20and%20Amazon,an%20S3%20bucket%20depend%20on%20several%20factors: