CloudFS Documentation

CloudFS is a Go filesystem abstraction library that exposes local and remote storage backends through a single cloudfs.FS interface. Callers can list directories, read files, create files, move, copy, rename, and remove entries with a consistent API.

Documentation Map

• Middleware Guide
• Driver Guide

Highlights

• Unified cloudfs.FS interface
• Direct driver construction and dynamic driver construction by name
• Middleware-based composition on top of existing drivers
• Support for prefix mapping, caching, rate limiting, compression, encryption, and custom path/file mapping

Install

1go get github.com/honmaple/cloudfs

Import only the driver you need:

1import "github.com/honmaple/cloudfs/driver/webdav"

If you want to create drivers dynamically through driver.New(), also import the aggregate registration package:

1import _ "github.com/honmaple/cloudfs/driver/all"

Quick Start

 1package main
 2
 3import (
 4	"context"
 5	"io"
 6	"strings"
 7
 8	"github.com/honmaple/cloudfs/driver/local"
 9)
10
11func main() {
12	fs, err := local.New(&local.Option{
13		Path: "/tmp/cloudfs",
14	})
15	if err != nil {
16		panic(err)
17	}
18	defer fs.Close()
19
20	ctx := context.Background()
21
22	if err := fs.MakeDir(ctx, "/docs"); err != nil {
23		panic(err)
24	}
25
26	w, err := fs.Create(ctx, "/docs/hello.txt")
27	if err != nil {
28		panic(err)
29	}
30	if _, err := io.Copy(w, strings.NewReader("hello cloudfs")); err != nil {
31		_ = w.Close()
32		panic(err)
33	}
34	if err := w.Close(); err != nil {
35		panic(err)
36	}
37
38	files, err := fs.List(ctx, "/docs")
39	if err != nil {
40		panic(err)
41	}
42	for _, file := range files {
43		println(file.Path(), file.Name(), file.Size())
44	}
45}

Core Interface

 1type FS interface {
 2	List(context.Context, string, ...ListOption) ([]FileInfo, error)
 3	Move(context.Context, string, string) error
 4	Copy(context.Context, string, string) error
 5	Rename(context.Context, string, string) error
 6	Remove(context.Context, string) error
 7	MakeDir(context.Context, string) error
 8	Stat(context.Context, string) (FileInfo, error)
 9	Open(context.Context, string) (File, error)
10	Create(context.Context, string) (FileWriter, error)
11	Close() error
12}

Notes:

• All paths are Unix-style paths such as /, /dir, and /dir/file.txt
• Open() returns a readable and seekable file object
• Create() returns an io.WriteCloser and must be closed explicitly to finish uploads or writes
• FileInfo extends io/fs.FileInfo with Path(), Type(), and ExtraInfo()

Creating Drivers

Direct Construction

1fs, err := webdav.New(&webdav.Option{
2	Endpoint: "https://example.com/dav",
3	Username: "user",
4	Password: "pass",
5})

Dynamic Construction

 1import (
 2	"github.com/honmaple/cloudfs/driver"
 3	_ "github.com/honmaple/cloudfs/driver/all"
 4)
 5
 6fs, err := driver.New("webdav", map[string]any{
 7	"endpoint": "https://example.com/dav",
 8	"username": "user",
 9	"password": "pass",
10})

From a JSON String

1fs, err := driver.NewFromString("local", `{"path":"/tmp/cloudfs"}`)

Validation Helpers

1ok := driver.Exists("s3")
2err := driver.VerifyOption("s3", `{"endpoint":"https://s3.example.com","bucket":"files"}`)

Notes:

• driver.New() and driver.NewFromString() depend on per-driver init() registration
• Validation errors currently come back in Chinese

Middleware Composition

You can layer middleware through cloudfs.New() or middleware.NewFS():

 1import (
 2	"github.com/honmaple/cloudfs"
 3	"github.com/honmaple/cloudfs/driver/local"
 4	"github.com/honmaple/cloudfs/middleware"
 5)
 6
 7base, _ := local.New(&local.Option{Path: "/data"})
 8
 9fs, err := cloudfs.New(
10	base,
11	middleware.PrefixFS("/project-a"),
12	middleware.CacheFS(&middleware.CacheOption{ExpireTime: 120}),
13	middleware.RateLimitFS(&middleware.RateLimitOption{
14		Wait:  true,
15		Burst: 20,
16		Limit: 1,
17	}),
18)

See middleware.md for details.

Common Operations

1ctx := context.Background()
2
3files, err := fs.List(ctx, "/")
4info, err := fs.Stat(ctx, "/file.txt")
5err = fs.MakeDir(ctx, "/new-dir")
6err = fs.Rename(ctx, "/file.txt", "new-name.txt")
7err = fs.Move(ctx, "/new-name.txt", "/new-dir")
8err = fs.Copy(ctx, "/new-dir/new-name.txt", "/")
9err = fs.Remove(ctx, "/new-dir/new-name.txt")

Driver Capability Matrix

See drivers.md for complete parameter and behavior details.