5

I couldn't find this anywhere online. How can I create a custom user defined function in cassandra?.

For Ex : 

CREATE OR REPLACE FUNCTION customfunc(custommap map<text, int>)
CALLED ON NULL INPUT
RETURNS map<int,bigint> 
LANGUAGE java AS 'return MyClass.mymethod(custommap);';

Where "MyClass" is a class that I can register in the Classpath?

Venky
  • 101
  • 1
  • 10

3 Answers3

2

I have the same issue, too. The custom class in UDF is support in cassandra 2.2.14, but not in cassandra 3.11.4.

Go through the source codes, cassandra 3.11.4 setup the UDF class loader with no parent class loader so that it have full control about what class/resource UDF uses. In org.apache.cassandra.cql3.functions.UDFunction.java, a whitelist and blacklist is used to control which class/package can be access.

For your issue, you should add the full name of MyClass into whitelist, and re-build the cassandra.

Cà phê đen
  • 1,883
  • 2
  • 21
  • 20
haishan
  • 21
  • 2
  • The allowed access packages/classes in the white list: "com/datastax/driver/core/", "com/google/common/reflect/TypeToken", "java/io/IOException.class", "java/io/Serializable.class", "java/lang/", "java/math/", "java/net/InetAddress.class", "java/net/Inet4Address.class", "java/net/Inet6Address.class", "java/net/UnknownHostException.class", "java/net/NetworkInterface.class" – haishan Mar 19 '19 at 04:07
  • "java/net/SocketException.class", "java/nio/Buffer.class", "java/nio/ByteBuffer.class", "java/text/", "java/time/", "java/util/", "org/apache/cassandra/cql3/functions/JavaUDF.class", "org/apache/cassandra/cql3/functions/UDFContext.class", "org/apache/cassandra/exceptions/", "org/apache/cassandra/transport/ProtocolVersion.class" – haishan Mar 19 '19 at 04:07
1

1. First build your java project that contains your class. Remember you have to add package name to your class.

Example : 

package exp;

import java.lang.Math;
import java.util.*;

public class MyClass
{
  public static Map<Integer,Long> mymethod(Map<String, Integer> data) {
      Map<Integer,Long> map = new HashMap<>();
      map.put(1, 10L);
      map.put(2, 20L);
      map.put(3, 30L);
      return map;
  }
}

After compile and build i have the jar test.jar

2. Copy the jar file to all cassandra node's $CASSANDRA_HOME/lib Directory

3. Restart All Cassandra Node

4. Create your custom function

Example : 

 CREATE OR REPLACE FUNCTION customfunc(custommap map<text, int>) 
    CALLED ON NULL INPUT 
    RETURNS map<int,bigint>  
    LANGUAGE java 
    AS 'return exp.MyClass.mymethod(custommap);';

Now you can use the function : 

cassandra@cqlsh:test> SELECT * FROM test_fun ;

 id | data
----+------------------
  1 | {'a': 1, 'b': 2}

(1 rows)
cassandra@cqlsh:test> SELECT customfunc(data) FROM test_fun ;

 test.customfunc(data)
-----------------------
 {1: 10, 2: 20, 3: 30}

(1 rows)
Ashraful Islam
  • 12,470
  • 3
  • 32
  • 53
  • I tried this. I am getting InvalidRequest: Error from server: code=2200 [Invalid query] message="Java source compilation failed: Line 1: exp.MyClass cannot be resolved to a type – Venky Apr 27 '17 at 17:37
  • Did you gone through all the steps i have listed ?? It should work, i have tested it in my `cassandra 2.2.4` – Ashraful Islam Apr 27 '17 at 17:42
  • Ok. Tried in Cassandra 3.10 and didn't work. I will try to debug – Venky Apr 27 '17 at 17:50
1

Just adding my 2 cents to this thread as I tried building an external class method to support something similar. After trying for hours with Datastax Sandbox 5.1 I could not get this to work as it couldn't seem to find my class and kept raising type errors.

My guess is that external JAR-based code for UDFs is not supported (see http://koff.io/posts/hll-in-cassandra/ and https://issues.apache.org/jira/browse/CASSANDRA-9892). Support for "TRUSTED" JARs is in planning stages for Cassandra 4. It might work in earlier versions pre 3.0 but I'm using the latest version from Datastax.

To work around this issue, I had to fallback to using a Javascript version instead (I was trying to convert a JSON string into a Map object).

While I realize Java UDFs perform better, the code I was testing was using Java Nashorn javascript support anyway, so using Javascript might not be such a bad thing. It does end up with a simpler one-liner UDF.

user3117823
  • 11
  • 2