git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Go SDK: Biquery and Legacy SQL


Legacy SQL was the default when the IO was written. IIRC Standard SQL changed the structure of table names and had some problematic corner cases in that regard, so it was easier to just stay with legacy SQL. Feel free to open a JIRA and/or take a stab at it.

Btw, you should be able to use time.Time if you register the type and bypass the type serialization.

Thanks,
 Henning

On Thu, Jun 21, 2018 at 5:57 PM eduardo.morales@xxxxxxxxx <eduardo.morales@xxxxxxxxx> wrote:
I am trying to read a column of type TIMESTAMP, this type is mapped by the bigquery client to time.Time.  Unfortunately, it is not possible to use time.Time structs because this type contains slices which are not supported by the beam Go SDK (run fine in the direct runner, but panic on dataflow).

The workaround I was seeking was to use SQL function UNIX_SECONDS() thus converting the timestamp to an int64. The problem now is that UNIX_SECONDS is not part of the legacy SQL.

Why brings me to my questions: Why is the Go SDK using Legacy SQL? Is there a technical reason for this? If I write my own bigquery transform will I regret it if I use Standard SQL?