Stack Overflow
Stack Overflow

Questions tagged [bigsql]

IBM® Big SQL is one of two connection environments to query data that is stored on an InfoSphere® BigInsights™ Hadoop cluster.

IBM® Db2® Big SQL is a hybrid SQL engine for Hadoop that delivers data querying for the enterprise. Use a single database connection or query to connect to disparate sources such as HDFS, RDMS, NoSQL databases, object stores and WebHDFS.

47 questions
2
votes
0 answers

TIMESTAMP column not interpreting correct value for ORC file in HDP3.1

As part of cluster migration we are copying ORC hdfs files from old cluster - IBM IOP 4.2 to HDP 3.1. Post migration we see TIMESTAMP column shows -1 hour in HDP 3.1. Similar question posted in - TimeStamp issue in hive 1.1 We cross checked TIME…
Debasis Maity
  • 21
  • 1
2
votes
2 answers

How to EFFICIENTLY query n records per category

To select N records per category one can do: SELECT category, category_id, value FROM ( SELECT category, value, row_number() OVER (PARTITION by category) as category_id FROM myTable ) WHERE category_id < N; The inner SELECT will first…
Zahra
  • 6,798
  • 9
  • 51
  • 76
1
vote
1 answer

JOIN LINES in BigSQL where PRODUCTs are the SAME

I am having this sample table below +---------+----------------------+ | PRODUCT | TYPE | +---------+----------------------+ | WIN 10 | Home | +---------+----------------------+ | WIN 10 | Pro …
Brunoxy1
  • 13
  • 3
1
vote
0 answers

Does column order matter in BIGSQL while creating external table over parquet files

In my spark code I am writing my dataframe as a parquet file on hdfs. Then I have created an external table by changing columns order over those parquet files in BIGSQL and after querying the table, it shows me following error. But If I query the…
Akash Janjal
  • 11
  • 2
1
vote
1 answer

How to find the difference in terms of days from YYYY_MM_DD formatted dates in bigsql?

I want to find the difference between two dates in YYYY-MM-DD format. For example, I have 2 dates 2018-10-31 and 2018-11-07. I want to find the difference in number of days between those two dates (7 days) in BigSql. I have gone through stack…
Vin
  • 515
  • 3
  • 17
1
vote
1 answer

Db2 Pivot function - fetch next statement

Under https://www.itjungle.com/2015/04/21/fhg042115-story01/ a good article on how to write an universal Db2 pivot method can be found. Executing the following code (be aware of 'end #' instead 'end ;'): SET SCHEMA = TESTSCHEMA; CREATE PROCEDURE…
matz3
  • 88
  • 8
1
vote
0 answers

pgadmin backup Failed (exit code: 500)

Trying to backup any postgresql database but got this error, right after pressing backup button. I need to make back up in pgadmin. I have tried many variants pgadmin 3,4 - same error. Now i am on bigsql's pgdevops , but same error. Maybe something…
Whats Myname
  • 41
  • 1
  • 9
1
vote
2 answers

Message: RSA premaster secret error. ERRORCODE=-4499, SQLSTATE=08001

I'm trying to connect to BigSQL on a BigInsights on Cloud cluster from an iPython notebook on Bluemix Spark as a service. My notebook looks like this: import os cwd = os.getcwd() with open('certificate', 'w') as f: f.write('''-----BEGIN…
Chris Snow
  • 23,813
  • 35
  • 144
  • 309
1
vote
1 answer

MySQL issue with NULL values

I have a table with fields: country_code, short_name, currency_unit, a2010, a2011, a2012, a2013, a2014, a2015. a2010-a2015 fields are type of double. How do I make a query which orders the results by average of fields a2010-a2015, keeping in mind…
user4666255
1
vote
0 answers

Null Pointer Exception from Hive

I am able to execute the below query successfully from HIVE prompt. SELECT * FROM DB1.TABLE1; But whenever I am trying to execute any other queries like below: SELECT COL1 FROM DB1.TABLE1 WHERE PARTITION_COL = 201211; java.io.IOException:…
Koushik Chandra
  • 1,565
  • 12
  • 37
  • 73
1
vote
3 answers

install IBM Big insights 3.0.0.0, the Big SQL component error and can not start

When I install IBM Big Insights Quick Start Editor 3.0.0.0, other components are correct except the Big SQL component that has an error: can not start the Big SQL Head Node and Big SQL Scheduler Nodes . When I　run ./start.sh bigsql in…
huiwenhan
  • 127
  • 8
0
votes
1 answer

Hive/Bigsql pandas float cast to integer with nulls into parquet file with pyarrow

I have an issue with pandas, pyarrow and bigsql. Basicly i want to read csv , write it to parquet and read parquet from db. But there is a problem while reading file from db. Pandas dtype and db datatype are not the same. There is table created in…
Chan
  • 23
  • 2
  • 6
0
votes
0 answers

Pandas sqlalchemy error after to_sql chunksize too much

Im facing a problem with pandas.to_sql(). My db : ibm db2 (bigsql) table : hadoop table stored as parquet Im connecting to bigsql and load some data from other db through pandas like this. read = pd.read_sql(text(item['load_sql']),session.bind) …
Chan
  • 23
  • 2
  • 6
0
votes
2 answers

SQL Query - Get the number from 2 columns, into a 3rd one

I want to build the 3rd column within a query I have, it's a select query. I tried some stuff like this: Also thought of regex... It's for big data, so I'm concered about being efficient. Also though of a case switch... I'm using big query…
user12146291
0
votes
2 answers

partition mysql table on a none primary key column

I have a table: +----+---------+----------+ | id | user_id | comment | +----+---------+----------+ Where column type is: id (bigint not null primary key autoincrement) user_id (bigint not null) comment (text) How can I partition this table on…
Daniyal Madmolil
  • 1
  • 6
1
2 3 4