0

I have a fairly large query running on Clickhouse. The problem is when running on localhost using cmd line it takes about 0.7 sec to complete. This is consistently fast. Issue is when querying from C# / HTTP / Postman. Here it takes about 10 times to return the data. (the size is about 3-4mb) so I dont think its a size issue.

I have tried to monitor network latency, but nothing to notice here. On the host it works like a charm, but outside it does not :(.... what to do.

I exptect the latency to be a few 100 ms, but turns out to be 7 sec :/

Morten V. Gade
  • 79
  • 8
  • how much rows and columns in your query resultset? clickhouse-client used native clickhouse protocol Did you use native clickhouse protocol driver in C# ? https://github.com/killwort/ClickHouse-Net ? – Slach Sep 06 '19 at 18:47
  • Did you check what is the elapsed time for query execution for C# CH Client? – MeteHan Sep 29 '19 at 22:47
  • The error is confirmed to be in the C# client and in the internal Http CH server. I've made a .net core webservice which uses the native CH cmd line and return the result here. This is super fast:). The developer of the Net Ado implementation also states that the .net could be rewritten to use io.pipelines to solve the issue mentioned – Morten V. Gade Oct 01 '19 at 05:48

2 Answers2

0

check timings with curl https://clickhouse.yandex/docs/en/interfaces/http/ https://stackoverflow.com/a/22625150

and compare local vs remote

CH HTTP usually provides almost the same performance as TCP and HTTP could be faster for small resultsets (like 10 rows)

Denny Crane
  • 11,574
  • 2
  • 19
  • 30
  • As I wrote it's for large results (20k) lines. I'm just stress testing it, so in real applications I don't know whether 20k lines would be something to return. Just wondering why it didn't scale – Morten V. Gade Oct 01 '19 at 05:49
0
Again. The problem is not the HTTP.
Example:

time clickhouse-client -q "select number, arrayMap(x->sipHash64(number,x), range(10)) from numbers(10000)" >native.out
real    0m0.034s

time curl -S -o http.out 'http://localhost:8123/?query=select%20number%2C%20arrayMap(x-%3EsipHash64(number%2Cx)%2C%20range(10))%20from%20numbers(10000)'
real    0m0.017s

ls -l http.out native.out
2108707 Oct  1 16:17 http.out
2108707 Oct  1 16:17 native.out

10 000 rows - 2Mb

HTTP is faster 0.017s VS 0.034s

Canada -> Germany (openvpn)

time curl -S -o http.out 'http://user:xxx@cl.host.x:8123/?query=select%20number%2C%20arrayMap(x-%3EsipHash64(number%2Cx)%2C%20range(10))%20from%20numbers(10000)'

real    0m1.619s

ping cl.host.x
PING cl.host.x (10.253.52.6): 56 data bytes
64 bytes from 10.253.52.6: icmp_seq=0 ttl=61 time=131.710 ms
64 bytes from 10.253.52.6: icmp_seq=1 ttl=61 time=133.711 ms
Denny Crane
  • 11,574
  • 2
  • 19
  • 30