gRPC remote services with generate command

GENERAL
1

Recently bndtools has introduced the -generate command…that allows one to run an arbitrary process to generate source code as part of a bndtools project build.

In this video, I show how this capability can be used along with Google’s protoc compiler to generate a complete remote service API from a proto3/gRPC service definition.

I also show how the generated service definition can then be easily turned into an OSGi Remote Service, and tested.

Please see: https://youtu.be/4-f4xQBlKr0

Scott

2 Likes
2

Thanks! Very nice, enjoyed it.

3

I enjoyed this too, I think it’s really elegant.

But as a beginner with ECF/OSGi remote services, I found something was happening with the Health Check example that was a bit counterintuitive. Consider the following workflow:

  1. Start consumer and impl.
    • Both components start up and activate. Consumer connects to remote service and successfully communicates.
    • So far, so good.
  2. In the impl process, stop the impl bundle (“stop 6”).
    • Consumer process deactivates its consumer component
    • impl component deactivates
    • impl bundle stops.
    • Again, so far, so good.
  3. In the impl process, start the impl bundle (“start 6”).
    • impl bundle starts.
    • impl component activates.
    • Consumer process activates its consumer component and the whole thing runs again.
    • Still so far, so good.
  4. In the consumer process, stop the consumer bundle (“stop 6”).
    • consumer component deactivates.
    • “IMPORT_UNREGISTRATION” log message.
    • consumer bundle stops.
    • Still so far, so good.
  5. Now’s when things start to get interesting: start the consumer bundle again.
    • consumer bundle start.
    • consumer component doesn’t get activated! (Expected: consumer component should activate as the remote service is running and available.)
  6. In the impl process, stop/start the impl bundle.
    • impl bundle & component restarts.
    • consumer bundle notices the restart and activates the consumer component. Again, as expected.

Now there might be a logical explanation for this behaviour. But as a newbie to ECF and OSGi remote services coming to this, intuitively I would have expected a restart of the consumer bundle to just work without needing to restart the server side. Can you shed any light on this @scottslewis ?

4

Hi Jeremy,

Thanks for pointing this out. Turns out it’s some incorrect behavior (bug) in the remote service consumer unget handling/cleanup as described here:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=574281

I’m working on a fix…fortunately the fix is very easy. i.e. just removing a couple of lines and I will be including this fix in ECF 3.14.22…which I will hopefully release early next week (I have a couple of small other issues unrelated to RS to deal with before releasing).

BTW…since this posting and video I’ve added quite a lot to the grpc healthwatch project templates. Specifically, I’ve added server, client, and bidirectional streaming rpc methods in the HealthCheck proto file, and created example implementations (in the impl and consumer project templates) that exercise these grpc streaming methods.

FYI, the protobuf code generation actually adds three protobuf plugins:

  1. the grpc plugin–so that grpc code is generated;

  2. rxgrpc: So that reactivex code is generated by 3. This is a public prococ plugin: reactive-grpc/rx-java/rxgrpc at master · salesforce/reactive-grpc · GitHub that generates code that uses the reactivex API for the grpc streaming methods…specifically Flowable (for client, server and bidi streaming) and Single (for non-blocking or blocking plain 'ol rpc). Flowable and Single are part of the java impl of reactivex spec: http://reactivex.io/. Note that the cool thing about this is that salesforce has implemented the reactivex api on the grpc streaming support so that one gets automatic flow control out of grpc.

Note: I’ve considered using other (other than reactivex) event-driven frameworks (other than reactivex…e.g. osgi’s event streams, java 9 pub/sub impl). The reason I haven’t done this yet is I don’t currently have the means to do what salesforce has done with reactivex and grpc…i.e. provide an impl of event streams or whatever api on grpc. If this were to be done, however, I would be able to easily enhance 3 to support and use it (like I’ve done for saleforce’s usage of reactivex).

  1. An osgi remote service generator (what I built: GitHub - ECF/grpc-osgi-generator: Service interface generator plugin for grpc). All this currently does is to create a (remote) service interface(s) from the protofile-declared service. Here is an example from the HealthCheck service: grpc-RemoteServicesProvider/HealthCheckService.java at master · ECF/grpc-RemoteServicesProvider · GitHub. Note the use of Single and Flowable (from reactivex api) for rpc arg and return values. This plugin can/could be enhanced in several ways, and I’m hoping to do so soon: OSGi and/or bnd/bndtools metadata could be generated for api, impl and/or consumer usage (currently in grpc project templates). OSGi version info could be supported (in protofile source), and other things like remote service/RSA metadata also (options from protofile potentially). And then there’s the enhancements from/for 2 also that could be done.

Thanks again for pointing out the problem.

5

The fix for the issue described in previous comment has been fixed and released as ECF 3.14.22

The fix is also now available as part of the ECF remote services bndtools workspace template:

1 Like