Remote EJB in Kubernetes

Question

I'm trying to setup a remote EJB call between 2 WebSphere Liberty servers deployed in k8s. Yes, I'm aware that EJB is not something one would want to use when deploying in k8s, but I have to deal with it for now.

The problem I have is how to expose remote ORB IP:port in k8s. From what I understand, it's only possible to get it to work if both client and remote "listen" on the same IP. I'm not a network expert, and I'm quite fresh in k8s, so maybe I'm missing something here, that's why I need help.

The only way I got it to work is when I explicitly set host on remote server to it's own IP address and then accessed it from client on that same IP. This test was done on Docker host with macvlan0 network (each container had it's own IP address).

This is ORB setup for remote server.xml configuration:

<iiopEndpoint id="defaultIiopEndpoint" host="172.30.106.227" iiopPort="2809" />

<orb id="defaultOrb" iiopEndpointRef="defaultIiopEndpoint">
    <serverPolicy.csiv2>
        <layers>
            <!-- don't care about security at this point -->
            <authenticationLayer establishTrustInClient="Never"/>
            <transportLayer sslEnabled="false"/>
        </layers>
    </serverPolicy.csiv2>
</orb>

And client server.xml configuration:

    <orb id="defaultOrb">
    <clientPolicy.csiv2>
        <layers>
            <!-- really, I don't care about security -->
            <authenticationLayer establishTrustInClient="Never"/>
            <transportLayer sslEnabled="false"/>
        </layers>
    </clientPolicy.csiv2>
</orb>

From client, this is JNDI name I try to access it:

corbaname::172.30.106.227:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote

And this works.

Since one doesn't want to set fixed IP when exposing ORB port, I have to find a way to expose it dynamically, based on host IP. Exposing on 0.0.0.0 does not work. Same goes for localhost. In both cases, client refuses to connect with this kind of error:

Error connecting to host=0.0.0.0, port=2809: Connection refused (Connection refused)

In k8s, I've exposed port 2809 through LoadBalancer service for remote pods, and try to access remote server from client pod, where I've set remote's service IP address in corbaname definition. This, of course, does not work. I can access remote ip:port by telnet, so it's not a network issue.

I've tried all combinations of setup on remote server. Exporting on host="0.0.0.0" results with same exception as above (Connection refused).

I'm not sure exporting on internal IP address would work either, but even if it would, I don't know the internal IP before pod is deployed in k8s. Or is there a way to know? There is no env. variable with it, I've checked.

Exposing on service IP address (with host="${REMOTE_APP_SERVICE_HOST}") fails with this error:

The server socket could not be opened on 2,809. The exception message is Cannot assign requested address (Bind failed).

Again, I know replacing EJB with Rest is the way to go, but it's not an option for now (don't ask why).

Help, please!

EDIT:

I've managed to get some progress. Actually, I believe I've successfully called remote EJB. What I did was add hostAliases in pod definition, which added alias for my host, something like this:

hostAliases:
- ip: 0.0.0.0
  hostnames:
  - my.host.name

Then I added this host name to remote server.xml:

<iiopEndpoint id="defaultIiopEndpoint" host="my.host.name" iiopPort="2809" />

I've also added host alias to my client pod:

hostAliases:
- ip: {remote.server.service.ip.here}
  hostnames:
  - my.host.name

Finally, I've changed JNDI name to:

corbaname::my.host.name:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote

With this setup, remote server was successfully called!

However, now I have another problem which I didn't have while testing on Docker host. Lookup is done, but what I get is not what I expect.

Lookup code is pretty much what you'd expect:

Object obj = new InitialContext().lookup(jndi);
BeanRemote remote = (BeanRemote) PortableRemoteObject.narrow(obj, BeanRemote.class);

Unfortunatelly, this narrow call fails with ClassCastException:

Caused by: java.lang.ClassCastException: org.example.com.BeanRemote
    at com.ibm.ws.transport.iiop.internal.WSPortableRemoteObjectImpl.narrow(WSPortableRemoteObjectImpl.java:50)
    at [internal classes]
    at javax.rmi.PortableRemoteObject.narrow(PortableRemoteObject.java:62)

Object I do receive is org.omg.stub.java.rmi._Remote_Stub. Any ideas?

If you cant do REST as you wrote, then I would put both apps in single container, and use localhost, to avoid all mambo jumbo with ports. Or at least single deployment/pod with two containers in the pod. Then they are on same host and you could use localhost. Also try to put `host="*"` in your iiopEndpoint. RMI/IIOP is not a container friendly protocol. — Gas, Nov 06 '20 at 10:18
@Gas thank you for your reply. Separate pods is exactly what I'm trying to accomplish - having a separation between two application tiers. Both apps in the same pod is where I started from. Also, I've tried `host="*"` but it doesn't work, ORB refuses to start because of this error: `Caused by: java.net.UnknownHostException: *: *: unknown error`. — nkuzman, Nov 06 '20 at 11:53
Have you tried solution from here: https://github.com/WASdev/ci.docker#optional-enterprise-functionality If you add `ARG IIOP_ENDPOINT=true` to your dockerfile, you will be able to set hostname via `env.IIOP_ENDPOINT_HOST`. Although I would still recommend running them in same pod ;-) and separate when you will rewrite to REST. — Gas, Nov 06 '20 at 15:21
@Gas Thanks again. I think I managed to call it, but now I have another problem. I've edited the original post. — nkuzman, Nov 06 '20 at 15:48

score 2 · Accepted Answer · edited Jan 15 '21 at 17:08

Solved it!

So, the first problem was resolving host mapping, which was resolved as mentioned in edit above, by adding host aliases id pod definitions:

Remote pod:

hostAliases:
- ip: 0.0.0.0
  hostnames:
  - my.host.name

Client pod:

hostAliases:
- ip: {remote.server.service.ip.here}
  hostnames:
  - my.host.name

Remote server then has to use that host name in iiop host definition:

<iiopEndpoint id="defaultIiopEndpoint" host="my.host.name" iiopPort="2809" />

Also, client has to reference that host name through JNDI lookup:

corbaname::my.host.name:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote

This setup resolves remote EJB call.

The other problem with ClassCastException was really unusual. I managed to reproduce the error on Docker host and then changed one thing at a time until the problem was resolved. It turns out that the problem was with ldapRegistry-3.0 feature (!?). Adding this feature to client's feature list resolved my problem:

<feature>ldapRegistry-3.0</feature>

With this feature added, remote EJB was successfully called.

Remote EJB in Kubernetes

1 Answers1