-1

Is there any way to set session parameters of presto in spark, while building a Dataframe out of it.

public Dataset<Row> readPrestoTbl(){
   Dataset<Row> stgTblDF = sparksession
                            .read()
                            .jdbc(dcrIdentity.getProperty(env + "." + "presto_url")
                            + "?SSL="
                            + dcrIdentity.getProperty(env + "."
                                    + "presto_client_SSL"), demoLckQuery, getDBProperties());
}

private Properties getDBProperties() {
        Properties dbProperties = new Properties();
        dbProperties.put("user", prestoCredentials.getUsername());
        dbProperties.put("password", prestoCredentials.getPassword());
        dbProperties.put("Driver", "io.prestosql.jdbc.PrestoDriver");
        dbProperties.put("task.max-worker-threads", "10");
        
        return dbProperties;
      }

The way I have set task.max-worker-threads this property is there any option to set session properties like, required_workers_count or query_max_run_time etc.

I also tried below options, but every time its says Unrecognized connection property 'sessionProperties'.

while adding in properties

 dbProperties.put("sessionProperties","task.max-worker-threads:10");

while loading in spark

.option("sessionProperties", "task.max-worker-threads:10")
Kalpesh
  • 694
  • 2
  • 8
  • 28
  • Does this answer your question? [Unrecognized connection property 'url' when using Presto JDBC in Spark SQL](https://stackoverflow.com/questions/68989208/unrecognized-connection-property-url-when-using-presto-jdbc-in-spark-sql) – ebyhr Jan 25 '22 at 00:15
  • No, that question for url property, which working fine with me.. mine was session properties.. which is bit complex that url – Kalpesh Jan 25 '22 at 10:39
  • Then, please share the all information without omitting (log, Spark version, Trino server version, code and so on). I can't reproduce with 350 JDBC driver. – ebyhr Jan 25 '22 at 12:32

1 Answers1

-1

Trino (formerly PrestoSQL) JDBC driver supports sessionProperties property.
https://trino.io/docs/current/installation/jdbc.html?highlight=sessionproperties#parameter-reference

Also, this is a blog post about the rebranding.
https://trino.io/blog/2020/12/27/announcing-trino.html

ebyhr
  • 1,537
  • 9
  • 15
  • I tried as explained in question but then also it is giving same exception: Unrecognized connection property 'sessionProperties' – Kalpesh Jan 24 '22 at 12:02
  • What JDBC driver version are you using? Also, there is two issues in a post. The first should be resolved my answer. The second is the different issue. – ebyhr Jan 24 '22 at 12:47
  • I am using latest version 350, and while creating spark dataframe it gives the exception sessionproperties as unrecognized property.. is there any syntax error or something else.. if you can elaborate what different issue it would be really helpful. io.prestosql presto-jdbc 350 – Kalpesh Jan 24 '22 at 13:20
  • Your question didn't contain enough information, so my answer was correct in your first post. Now your question is duplicate of https://stackoverflow.com/questions/68989208/unrecognized-connection-property-url-when-using-presto-jdbc-in-spark-sql. I would recommend asking in the community Slack if you want to debug interactively. https://trino.io/slack.html – ebyhr Jan 25 '22 at 02:59