Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How does this work with BigQuery Managed Writer #340

Open
newtonnthiga opened this issue Aug 9, 2023 · 0 comments
Open

Question: How does this work with BigQuery Managed Writer #340

newtonnthiga opened this issue Aug 9, 2023 · 0 comments

Comments

@newtonnthiga
Copy link

Migrated recently to use Bigquery Storage Write API(https://cloud.google.com/bigquery/docs/write-api-batch#go), and was wondering how this package works with the new API. Infer schema works perfect but it conflicts with the Proto Descriptor which is now needed when writing to pending streams

        // We need to communicate the descriptor of the protocol buffer message we're using, which
        // is analagous to the "schema" for the message.  Both SampleData and SampleStruct are
        // two distinct messages in the compiled proto file, so we'll use adapt.NormalizeDescriptor
        // to unify them into a single self-contained descriptor representation.
        m := &exampleproto.SampleData{}
        descriptorProto, err := adapt.NormalizeDescriptor(m.ProtoReflect().Descriptor())
        if err != nil {
                return fmt.Errorf("NormalizeDescriptor: %w", err)
        }

        // Instantiate a ManagedStream, which manages low level details like connection state and provides
        // additional features like a future-like callback for appends, etc.  NewManagedStream can also create
        // the stream on your behalf, but in this example we're being explicit about stream creation.
        managedStream, err := client.NewManagedStream(ctx, managedwriter.WithStreamName(pendingStream.GetName()),
                managedwriter.WithSchemaDescriptor(descriptorProto))
        if err != nil {
                return fmt.Errorf("NewManagedStream: %w", err)
        }
        defer managedStream.Close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant