Category Archives: jammazwan

Jammazwan – Camel keeper

Bare Bones MicroServices Camel

appwriter_logo

The purpose of this article is to help you get through some very quick and concise examples of microservices using Camel.
This might also be called a “bare bones” approach.

Before we start, I’ll try to clarify some potential confusion.
This may take more time than reviewing the code itself 🙁
Skip through, accordingly. The meat is in the bottom half.

Micro Service, Maximum Confusion:

Microservices are best characterized by what they are not. Which leads to confusion, because a microservice which is not anything, also fails to exist. So it has to be something. Stay with me here.

As soon as a microservice becomes something, the arguments begin. “I’ll use REST,” so someone blogs “Microservices suck because REST sucks.” Then you have to explain to the hater that microservices don’t require REST, which may only confuse him, because most, in fact, may use REST APIs. And so on, through every decision point. Exhausting.

It was tempting to name this blog “Haters Guide to Camel Microservices” but that would only serve my own frustrations with the human side of this puzzle.

Microservices Defined In This Context

In this context, I will approach microservices as an attempt to eliminate the ball and chain of required infrastructure. No hosting Tomcat or service bus … or …. whatever might be getting your dander up. It can be a long list. Every technology has it’s own haters. Go to any conference. Embarrassing, at least for the technology agnostic.

The approach here will be to eliminate everything possible – the bare bones approach – because that makes Camel microservices easiest to learn.  This is actually a big win, because it is easier to learn how to add stuff back in, than learn what can be taken away after it is added. Or, so it was for me.

Next, a Spring Boot project will make up the second service. By adding the absolute minimum Camel code to this, you will quickly see the delta. Not much to see.

Debating Society Disclaimer

Given the impressive debating society around all things microservices… Please do not use these two projects as part of this debate.  These example projects are learning tools only, and have no merit, other than a place to pick up how to information quickly.

Why So Difficult To Find a Bare Bones Example?

The last piece of confusion to resolve is this:
If Camel requires nothing more than a bare bones setup, why is there almost no prominent online example of same? Excepting for unit test projects, which might only confuse the novice? Even Claus Isben’s own Spring Boot Example seems to include layers of opinionated infrastructure, which can obfuscate how bare bones a Camel service might be.

You won’t have to think hard to answer this question. No one really believes you would ever want to deploy a bare bones service. “You’re at least going to need this and this and this.” Great for teaching this and this and this, not so great for learning the essential minimum.

About the Routes

All routes in this set of examples try to do as little as possible, other than prove that they are running.

It is assumed that you would  or have learned about Camel routes elsewhere, and would insert your desired routes herein. One such modification would be to provide a serviceable camel endpoint such as REST or file or jms or  … which neither of these services has 🙂

Two of Quadzillion Deployment Methods Provided For:

So many options! But not here. I run both projects with either of

  1. Shell script (java -jar etc)
  2. Dockerfile

What is not included? It is assumed that the reader either has his favorite microservice vendor – i.e. AWS – or he is being assaulted daily by many vendors who would like to be his favorite. Nearly any could such platform could be consumed, with 1 or 2 above, and a little extra account metadata.

Each project renames the jar which maven produces to make it easier to point the shell script or Dockerfile. This is done using <finalname> in the pom.xml.


Series Advertisement: Jammazwan

“Jammazwan” is Hindi for “camel keeper”.
This blog is part of that series.

Me, a jammazwan? another Camel Keeper: TL DR?     about:
pete jammazwanPhotoSmall jamzVid1

If you are impatient or in a hurry, the 2 minute video might explain Jammazwan in the shortest amount of time.


Bare Bones Camel MicroService:

Github: BareBonesCamelMicroService

Little more than a main and a route. Just to show you how skinny it can get. Too skinny to be representative of a real service.

myMainClass

This is pretty much the only code in the project. Of these 24 lines, only 3 make this a Camel microservice:

Main main = new Main();
main.addRouteBuilder(new MyRoutes());
main.run();

The rest of this code is just imports, a main(), and the RouteBuilder with a 1 line route, which you would expect in anything Camel, microservice or not.

Note that I also had to add the maven-shade-plugin to the pom.
I left some sample shell scripts to launch it as either a  straight jar, or a docker image. Adjust scripts as needed to match your own file placements.

Other than that, the project is nearly empty. And now you know how to create the skinniest microservice imaginable.

Note also that no port is exposed in the Dockerfile, as this service has no endpoints.

Bare Bones, except adding Spring Boot

Github: SpringBootBareBonesCamelMicroService

What would be the absolute minimum number of steps required to use Spring Boot as it comes right out of the box?
This project answers that question.

Here’s a screenshot of how I generated it. Spring makes start.spring.io useable for anyone to use:

startspringboot

So far, no camel code whatsoever, other than naming a dependency in the pom.xml (lower right). A 30 second operation, start to finish.

Look below, and you’ll see exactly what was added after downloading. Notice that the three highlighted lines are literally same as those in the BareBones example above. The route is the same, and the primary difference is the different wrapper to run those three lines from above, only in the boot application.

addToSpringBoot

As with the previous project, the jar name was adjusted in the pom, and shell scripts and Dockerfile were added for deployment/running.

I’m hoping by now you would agree, there isn’t that much work to a Camel microservice, if you want to go bare bones. And repeating from above, any cloud vendor would be happy to take it from there. You can repeat this entire process in less time than it takes to read this blog, once the process is familiar.

Now That You Have Learned The Minimum:

You may now add back in whichever pieces of infrastructure technology that don’t get your dander up. I’m sure you have your own favorite list.

Most will make the right choice, and head towards something like Claus Isben’s adaptation of Spring Boot. If you do that now, you would at least be well informed. It’s not quite as well documented, but now you know the essential minimums.

Or, better yet, take full advantage of fabric8 and/or Openshift!

Go for it!

Be Glad I Didn’t Write Apache Camel

appwriter_logo
If I had written Apache Camel, instead of my personal heroes – James Strachan, and Claus Isben – I would have completely screwed it up.

A nightmare. Allow me to clarify:

Explicit Everything – a Religious Belief:

In my world, the very word “implicit” is synonymous with Evil.

Implicit languages, implicit conversations, implicit meanings – these are all destined to confuse and sometimes even disrespect people. Or at least, disrespect their time.

Everything implicit, to me, is like an “in group” where only the initiates know the secret language. You can join the group, but only after you learn some implicit meanings. To heck with that.

Implicit languages, for example, are a little faster to code in, harder to tool, and much harder to learn than explicit languages. Or at least, that’s my perspective. Java is dumb as a box of rocks, compared to JavaScript – and that’s why I like it better. A reasonable example that favors the explicit.

I could go on and on, but it would only be more of the same. Law, governance, politics, culture, implicit vs explicit is a key issue everywhere, and a vast source of mischief.

Camel Works. Because of Generics. Implicit everything…

My worst nightmare is projects like Camel, where I cannot use explicit hierarchical topologies to walk a tree of types or functional options.

Camel uses generics, casting, and “type converters” to allow what amounts to an “any to any to any” kind of message passing. All without your explicit knowledge. If implicit was evil, this is evil on steroids.

But instead of being evil, it just works! Implicit is what allows the vast, and growing, array of Camel components and other usages to just work. It may make me appear lazy, (I am), but the extra work of testing at runtime really isn’t that bad, after making the mental adjustment.

Topologies as a Police State

Compare that to my chosen approach: Everything is a finite, known, hierarchical topology.

Except who makes the topology? Is it me, you, or ??? Now, we have a classic case of the Police State to the rescue.  I would probably have installed myself as the policeman, and Camel would never have grown beyond my ability to police the topology of types and functions. That would be amount to 1% of it’s current scope.

But that’s OK, because it would never have grown anyway.  Nobody even likes participating, where a police state is involved.

Conclusion

I have ragged on and on about Apache Camel and how needlessly hard it is to learn. I am not completely off, in this assessment.

But the point is that there is some magic to this design. It’s difficulty is in it’s brilliance. And it isn’t even that hard, you just have to get used to testing well at the runtime level.

No. That isn’t so hard.

 

Camel CXF – X__project Supplement

appwriter_logo

Who:

This blog should be interesting only to someone actively attempting to get their first camel-cxf project working. It is a companion piece to a progression of 4 example projects, and the docs on each.

  1. Code first CXF – simple.
  2. WSDL first CXF – simple.
  3. Code first CXF – complex objects.
  4. WSDL first CXF – complex objects.

It is notable that none of the previous Camel example projects required anything close to this kind of documentation, or explanation.

Main Points:

If you are impatient, like me, you may find that getting to number 4, above, is needlessly difficult, the first time. Especially aggravating, given that number 4 is the only variant that commercial/corporate applications might accept.

Then, when you get it all done, you might wonder what the fuss was about. Maybe this blog and these projects will get you there without spending much time.

The idea is to be able to blast through each of these very quickly, picking up additional steps/dependencies with each one.

The Sequence:

  1. Code first simple – basic dependencies, server, testing, tooling.
  2. WSDL first simple – add WSDL first steps, code.
  3. Code first complex – adds objects for input and output, processing, and testing variants.
  4. WSDL first complex – adds a merge with your own entity code.

Again, once you know the above steps you’ll wonder what all the fuss was about. Until then I found lots of ways to guess wrong and waste lots of time.

Specific to WSDL-First:

If you are new to SOAP and WSDL first in general, there may be some confusion about the generated code step. Since there may be some options, here is what I found successful.

  • Just running mvn compile is enough to generate the wsdl2java step. It’s will place it wherever you told it to in your pom, or in this case target/generated/src/main/java.
  • Wasn’t sure if I should be making this a legit source folder? Or copy the code to my src/main/java? I chose the latter, see next step.
  • When you copy it to your source folder, there still may be plenty of changes you have to make by hand. For example, your complex input and output object entities may already be in your codebase, and the wsdl2java generates you new entities. So in that case, you would have to merge the generated into the previous entities and then modify the rest of the generated code to point to those, and not the generated. You then delete the generated.
  • You may also have to comment out your wsdl2java plugin code in the pom.xml as soon as you have copied over and modified any generated code. This may or may not be obvious, but will make itself quite obvious if it becomes an issue.

Consuming WSDL from CodeFirst

One possible source of confusion can be quickly clarified by acknowledging a cheat that was used on the two WsdlFirst projects.

One of the most significant challenges to a WSDL first project of any type is the creation of the initial WSDL. This file is a complex 5 part file with many cross-referencing parts. Not for sissies like me, putting together silly example projects.

What was done on these example projects was to create the WSDL using the code first project, which creates a wsdl and publishes it to the browser as part of it’s functionality set.

This WSDL was then hand copied to the correct folder in the WsdlFirst projects.

A side benefit was that the WsdlFirstComplex and CodeFirstComplex projects could then also share entity classes, but that is just a side effect. In this case, the shared entities were placed in the jammazwan.shared project. Both WsdlFirst and CodeFirst projects share the same test code, using this approach.

Running Your Test as a Server (WTH?)

Problem: Jammazwan example projects are for learning Camel, not production deployments. Server and deployment code is intentionally excluded so you can isolate only what you are learning. So what happens in a case like this, where we actually need a running server to test with during development?

Solution: A quick, and ugly, manual hack is provided in our test code. I cringe, but it works great for the learning process.

A more realistic solution begins to outline itself in my Camel Microservices blog, though that is a bit off topic for this article.

Sequence:

  • Write your project to test with a normal test, such as template.requestBody(), like any other camel route.
  • Once it’s working, test it in the real world using something like SoapUI. To do this, uncomment the hack line

    Thread.sleep(Long.MAX_VALUE); Ugh.

  • You may now run the test, and the test basically hangs. Whola! Your server is operating. Even if you are wincing in disgust.
  • From your camel-ctx.xml file, copy the url. Such as http://localhost:9876/passThru/
  • Append ‘?wsdl‘ to this line. You now have a wsdl url of http://localhost:9876/passThru/?wsdl
  • Open the wsdl url in your browser. My chrome browser shows the xml from the wsdl file.
  • Now you can take this same wsdl url to your SoapUI tool, and test the actual working server with actual working xml. I won’t cover this here, as it’s assumed you already have this skillset.
  • Now you can kill your running test, which never finished, and comment back out the Thread.sleep(). Your manual dev test is complete.

Remaining Documentation:

Each of the 4 cxf example projects is documented in their respective NOTES.md

The NOTES.md files each focus on a diff between that, and previous artifacts. So 4 isn’t so much of a standalone, as a diff between it’s artifacts, and the previous. Etc. Other x__projects are standalone, not these 4.

  1. Code first CXF – simple –  NOTES.md
  2. WSDL first CXF – simple-  NOTES.md
  3. Code first CXF – complex objects –  NOTES.md
  4. WSDL first CXF – complex objects – NOTES.md

 

Jammazwan FAQ

appwriter_logo

  • You said my x_project was complete working software! How do I deploy it?
  • How to import into my IDE?
  • Are goals of example projects contradictory?
  • Why is my x_project written with spring/java configuration?
  • What is an x_project?
  • Why the jammazwan.shared dependency?
  • What is this x… prefix?
  • Generated or not? README.md vs NOTES.md
  • What does jammazwan.maker have to do with any of this?
  • Complex project warnings WTH?
  • Local v Remote JMS, DB, FTP
  • X__Projects vs jammazwan.100? WTH?
  • Uh, this isn’t a comprehensive list of example projects.

You said my x_project was working software! How do I deploy it?

x__projects are designed to isolate routing functionality and protect the user from confusing, deployment related code. They are run with unit tests. “Working,” in this context, is not intended to imply that it is deployable on a server.

 

The routes you would learn about from these isolated examples may be deployed into any server or service as appropriate. One such approach is outlined in another blog, but there are also vertical stack approaches such as Fuse which are amply documented by Red Hat.

Fabric8 or Openshift are also great deployment options, but again, provide their own learning curves.

How to import into my IDE?

You are encouraged – but not required – to open up your IDE into a new workspace and do all jammazwan work in a fresh workspace. This has nothing to do with jammazwan specifically, but is helpful practice whenever experimenting with new tech.

If you use Eclipse, then Import > General > Existing Projects into Workspace > Next > Browse > … then follow the prompts.

Other IDEs, or even Eclipse as an alternative to above, will import any project which has a pom.xml file. You may follow this normal process to import your jammazwan project into your IDE.

Are goals of example projects contradictory?

Jammazwan example projects are all about:

  • Driving adoption to increase learning speed.
  • Isolation of functionality to increase learning speed.

Isn’t that a contradiction? Don’t you need to pick one, instead?

Yes, it’s a contradiction. Maximizing the tension between the two is where learning gets real. Drive adoption and you create a huge desire to learn more, really fast. Isolate functionality and you make Camel easy to learn. Together, it’s a magical combination.

The tension between the two does create other problems, though. These are great problems to have.

Why is my x_project written with spring/java configuration?

There are so many language and configuration combinations available to the author of a Camel project, there could be no canonical, or even “best” choice.

Instead I opted for what seems to be the most common and powerful configuration choices. The choice of Java for it’s fluent API is the most important, because it is the most powerful.

What is this x__project naming convention?

One of the major challenges of isolating functionality in example projects is that now you have too many of them.

x__project is often used in the docs as a name for

see below: “What is this x… prefix?” if you need more info on this.

Why the jammazwan.shared dependency?

Some, but not all, x__projects require jammazwan.shared as a dependency, as indicated in their respective README.md

X__project design tries to include only what is unique to this example, so you can focus only on what is required to demo the MyCoolCamelComponent feature in isolation. The single shared project includes dumb artifacts that shouldn’t be unique to a single x__project, such as:

  • Beans used as value objects for all example projects
  • pom.xml dependencies of a more general nature
  • .csv and other source files for use as data sources
  • generic utility functions not specific to a single project

So you sometimes have to git clone jammazwan.shared as a sibling first, then run mvn install, before installing your project.

If you want to replicate the functionality of your project into your own stand-alone project, you will need to pull some dependencies and code from each.

Another pattern I could have followed would be to create jammazwan.shared as a parent project. This pattern has been horribly abused in recent years, (IMO), creating insanely nested and hard to decipher project structures, so I just avoided it entirely.

What is this x… prefix?

The prefix x__ is used for maintaining sort order in a directory, providing packaging, and is otherwise meaningless. The x stands for example.

A side benefit is that I do a lot of work in a bash shell, so being able to type cd ../xao [tab] saves me a lot of time, and is easier to remember than a lot of project names.

Generated or not? README.md vs NOTES.md

Every project has a README.md which is generated from here, and never hand modified.

Some projects also have a NOTES.md which is entirely created by hand, and contains information which is not in the metadata. Go there first if you have any needs that are not answered in the opening page README.

What does jammazwan.maker have to do with any of this?

All x__projects are created with a generator. Interesting notes about the generator, if you care:

  • Written entirely in Camel
  • Can create, and update, any project.
  • Uses Velocity component
  • Open source, use it for anything you wish.
  • Not in any way documented or supported.

Complex project warnings WTH?

Xab and at least one other project start out by advising you to go away. Kind of silly.

The whole point of x__projects is isolation of just one small thing. These projects break that implicit contract. So instead of learning just one thing you are confronted with way more than you can absorb, possibly doing more harm than good.

I’m still not sure why they are here, but some day I might come up with a better strategy and explanation.

Local v Remote JMS, DB, FTP

to be documented later, if ever.

X__Projects vs jammazwan.100?  WTH?

Is Jammazwan about x__projects or jammazwan.100? Don’t they have different goals in mind?

  • jammazwan is 90% about example projects
  • jammazwan.100 is for advanced work

Uh, this isn’t exactly a comprehensive list of example projects.

 

Pull requests welcomed.

Jammazwan – For Hire

appwriter_logo

Camel Keeper: Jammazwan

“Jammazwan” is Hindi for “camel keeper”.

Me, a jammazwan? another Camel Keeper: TL DR?     about:
pete jammazwanPhotoSmall jamzVid1

You may have already read my post on making Camel easier to learn.

Contract Camel Work

Please contact me to supplement the work that your own team might be doing with Apache Camel or JBoss Fuse.

I can bring an official Red Hat consulting shop with me as your go-between,  even a larger team – if required.

Smaller contracts by 1099, or even W2.

pete dot carapetyan at gmail dot com

Months To Learn Camel? Alternatives:

appwriter_logo

This article diagnoses a complex problem and offers time saving tips. To skip the tips and diagnosis, and go straight to a simpler solution, click here.


Apache Camel is an easy and fast way to do otherwise hard tasks – why can it take so long for some of us to learn it deeply? It doesn’t have to.

A random list of Camel learning challenges:

  1. Welcome to Camel, where every dot can be a new research project!
  2. With so many ways to do anything, first, use up all your time budget just researching which approach to use!
  3. Type converters, generics, and overloaded methods are your friend. Except when you are learning.
  4. It’s only faster to code than native xyz code, when you also know enough of xyz to know what it’s faster than, so you can configure camel properly.
  5. Snippets help the documenter, perhaps not as much the learner.
  6. Fuse, fabric, other deployment/run options are super! But have almost nothing to do with Camel except by association, and as consumers and servers. Confusing.
  7. It’s normal to read docs, instead of  the tests in the source code.
  8. To learn anything, you need real data to learn with. Another distraction: You need to learn how to acquire the data, first.
  9. The need for realistic examples that you can relate to (broad) is always in direct conflict with realistic examples that show you how to do things (narrow).
  10. Project setup and deployment is a separate obstacle, and a distraction for learning how the fluent API works.
  11. Straight Math: (Number of possible components) X (number of configuration options for each) X (number of ways to wire them together) = (brute force memorization barely even an option).
  12. There appear to be no consistent conventions. This is an illusion, but a really convincing illusion. Everything you learn with one, may seem completely different, with another endpoint or component. Maddening.
  13. Version congruence: This can waste much of your time. Know why.

Look, as a developer, none of the above even matters, until it starts wasting your time. Then, it matters a lot. Above all, please respect my time.  Here’s what I learned about how to reduce wasted time, with Camel.

Is Learning Instinctual?
Camel, and Cognitive Load

Scientists sometimes observe correlations between intelligence and a tolerance for ambiguity. This article will attempt to harvest your own best capabilities for learning Camel, by identifying those areas where cognitive load feels like it might blow your circuits.

Where this gets interesting is when you inject time into the learning model. Put an oversized Cognitive Load into the model, and time increases way out of scale. This becomes real when you only have a weekend to study something, and a Cognitive Load entirely prevents you from learning that one “xyz” task that you set out to master. Failure vs success, with the next available weekend coming up in 7 weeks.

Anticipate properly, and you can absorb patterns that might otherwise seem bizarre, especially to those of us who prefer a more deterministic feeling approach than Camel sometimes provides.

Sal Khan, founder of Khan Academy writes beautifully about the process of learning, especially as it relates to the importance of gaining context first, and sequentially. From that perspective, the simple task of identifying landmines to the learning process can assist us the learner in getting those out of the way, before they frustrate you.

Acknowledging Claus Isben

Perhaps your best, and worst, friend is Claus Isben

  • Seems to have written Apache Camel, almost single-handedly (? true?)
  • Has written the primary go-to book “Camel in Action”
  • Has written most and best answers on  Stackoverflow
  • Has been a best presenter, wherever he goes, including youtube.

Claus suffers from the same handicap that anyone who has already learned Camel suffers from. It’s hard to remember which foundational understanding is taken for granted in any teaching context. So his best efforts might sometimes assume a necessary understanding which you lack. This is not avoidable, not by Claus, nor by any human.

Claus Isben

Definition: “Learning Camel”

“Learning Camel” in this article should means a depth beyond the ability to run working projects. It’s easy to use Camel even without learning it to this depth – and I have done so.

But when it’s time to really learn Camel deeply, here is what that could mean:

  1. Enough Camel skills to pass the EX421 Certification Exam
  2. Enough Camel skills to be fast when writing routes, setups, etc.
  3. Enough Camel skills to be authoritative as an architect or designer, knowing how and when to use Camel in every situation.
  4. Enough Camel to pass an interview well.
  5. DOES NOT include edge cases, such as how every single obscure component might work. That can be learned as needed.

Definition: “XYZ”

“xyz” in this article is defined as whatever it is that you are attempting to learn at one given moment. A component, an endpoint, a transformation, configuration, or … whatever that is, “xyz” means something as narrow as possible, lest you get lost in the sauce.

Toolkits:

I don’t usually have to tool up so heavily to learn a new skillset. But most skillsets aren’t like Camel. These were in my toolkit:

  1. The book: Claus Isben’s Camel In Action is one great choice.
  2. [optional] The code from the book. Perhaps even more helpful than the book itself, at times. See below.
  3. The entire Camel codebase ($git clone …). More below, but not optional.
  4. Workspace usage and configuration. See below. This is can slow you down if you approach it the normal way.
  5. The Camel site, and Stackoverflow.

Strategy: Bite Size

With Camel, you can eventually do anything. Repeating for emphasis, the trick to is learning it faster is the sequence that you learn it in.

So,

  • before you read the book.
  • before you run your first examples.
  • before you commit to a course of action.

Before you do anything, learn the trip-wires that can keep your brain spinning needlessly. Once you do that, you can can learn sequentially, and it may be a much faster process.

Land Mines, Trip Wires

There are at least four ways that Camel designs it’s API to be any-to-any, and they all have the side effect of obfuscation or confusion.  Candidates for Cognitive Overwhelm. These include.

  1. Type Converters
  2. Generics and overloaded methods
  3. Configuration
  4. Multi-everything.

Sure, you can often do a lot of Camel without worrying about these trip-wires and land-mines. But that’s not the same thing as really learning Camel. See Definition: Learning Camel above.

Defense against Type Converters

Every time you type a dot (.) in the Camel fluent API, you may be engaging a helpful Type Converter, acting in your behalf, but without your knowledge.

WTF? “I thought that the message was a [….] at this point.” But no, it’s a [….] and Camel just did this conversion to help you. The fact that it doesn’t feel so helpful when you have no idea what is going on under the covers is the point here. You have to get used to this, and usually the only defense is to engage in some form of forensics to learn what Camel did “for you.”

Such forensics are beyond the scope of this article, but one key defense is below, in red.

Defense against Generics and Overloaded Methods

Code-completion in the IDE gives you the idea that  the fluent Camel route API uses static typing to guide you along your way. Cool!

But wait! It specifies most arguments using java generics to let you pass all manner of good stuff around. It’s statically typed, sure, but then you learn that all those static typed arguments are little more helpful than unfamiliar marker interfaces! How helpful does that feel? It often feels impossible (to me anyway) to figure out exactly what and how I am expected to be passing arguments into the next part of the fluent API. This, combined with overloaded methods, and a combination of 6 ways to do anything… one unfamiliar fluent API call can take you a whole morning to research, especially if you look in the wrong places first, which I have done a lot of.

Reminder: All this functionality is only here to help, even if it doesn’t feel that way when you are scratching your head in cognitive overwhelm.

As above with Type Converters, it takes forensics to figure this stuff out. One primary defense is below, in red.

Defense Against Configuration Challenges

Routes and endpoints change often, duh. OTOH, project and component configuration remain relatively static, especially on a single team, but they are only static once you achieve them! Learning Camel means that instead of learning about routes and endpoints, instead you are learning about xyz configuration, and project configuration, and that can take a some time.

There are many primary defenses against the challenges of learning project and xyz configurations.

  1. Learn configuration separately from the routes and endpoints, if at all possible.
  2. Start with example projects that are as narrow and close to the configuration you need, as you can get.
  3. See list below on what to “Avoid These As Long As Possible”
Defense against Multi-Everything

 

I can offer so many examples to illustrate the scope of this multi-everything problem, I just have to pick one:

In prepping for the exam, I was attempting to build an exhaustive list of splitting situations so I could knock any one that came up in 15 minutes. So of course, one of the splitting use cases involves xml. But starting the research I instantly came upon not one, but many ways to split xml. Too many to go over here. This research alone involved many different sources including grepping camel source code, and several books and reference docs.

So I built the capability on xpath, since that’s where most of the examples I found were. Then days later, while building examples of string splits, I happened upon yet another source of information deep within the camel site. There I learned about tokenizeXML and xtokenize built for special use cases where xpath doesn’t work as well. So now it’s back to the drawing board, and I am building 3 different sets of examples for splitting xml, not one. Total time invested is too embarrassing to reveal here.

I offer no definitive approach to resolving this TMI nightmare, but I do offer some heuristics.

  1. Unprofessional? I literally have to coach myself through the unprofessional feeling I get, when I can’t investigate every single approach first, before committing to a course of action. Yes, I’d really like to know which one works the best and in which circumstances, but that would take too much time, in Camel land. 🙁 Give yourself permission to not learn everything, and hope and pray that the approaches you decide to not learn aren’t on the exam, or in an interview.
  2. Occam’s Razor: Stick with the simplest approach first, or the first that you find first, or the one that you get working first. You will probably find a better way later 🙁
  3. Quick Surveys and Memorization: Sometimes I can coach myself with word-tracking that goes like this. “Well, the book gives me these 6 different approaches to writing a transform, but I don’t have time to learn them all in detail right now. So maybe just memorizing the names of these 6 approaches would help me in an interview.”

If you love to code, and you contract out as an architect, and you have an obsessive personality, saying no to all these super great possibilities may be the hardest part about learning Camel. And the most important.

Runnable Code is Preferable to Documentation

A helpful strategy is to opt for running example code over documentation, or at least, sequentially. Documentation is invaluable, but usually makes much more sense after getting an example running.

Especially important is sparse running code. The exact minimum is the best, especially in Camel where it’s so easy to go off on tangential concerns such as upstream or downstream endpoints.

I believe in documentation, so if you’re having a hard time with the idea of ignoring documentation, the way I get my head around it is to imagine how insanely difficult it would be to write adequate documentation for something as flexible as a Camel endpoint or component. One component could be an entire chapter, or even book, if you were meticulously thorough about all the incoming and outgoing options, with examples. Ridiculous.

Budget Time for Project Setups and Data Feeds Separately

An old joke about Regular Expressions: “There was this guy who had a programming problem that required a Regular Expression. Now he has TWO programming problems.”

May be odd to say this, but Camel project setup requires a different kind of mental state, when in cognitive overwhelm. Get that out of the way before thinking about your routes much, or trying to learn “xyz.”

And setting up your data feeds too. Get that done before trying to learn how to do xyz with the data. One more distraction out of the way. At this point, you’ve got your data ready to process, all you have to do is xyz…

The nice thing about getting your setups done separately is you can often reuse such setup projects.

I keep a “shared” project as a dependency. It’s got enough of the basic data feeds and configuration dependencies to give me most of what I need. Then I set up a separate xyz project just for the delta – that part that corresponds only to xyz.

See jammazwan.xyz

10 Minute Reps

Cognitive Load is a fancy name for whatever you actually have to think about. So if you need to focus on learning xyz, everything before needs to already be a part of your muscle memory.

Doing reps on the setups is one approach to dumping that portion of the learning into your muscle memory.

Muscle memory in Camel is easier than you might first guess, with another reps. Jammazwan.setups is designed for gaining such muscle memory.

Avoid These, For As Long As Possible

When possible, there are super excellent aspects of Camel that you should just plain avoid, until you are out of the basic learning phase. Learn Camel first.

  1. Deployment options.
  2. Expression language options. (Go with default).
  3. XML routing options, blueprint, spring xml, etc.
  4. Arcane components such as Avro or Undertow or …

It’s not that you shouldn’t learn them, just don’t try to learn them at the same time as you are picking up xyz, or configuration basics, or …

Your strategy here might be compared with a child eating dessert. You have a delicious treat waiting for you, after you eat the vegetables.

Accommodation: Meaningful Example Projects

In direct opposition to the need for sparse, or narrow, sample projects (see above) there is a need to keep your brain interested and engaged. Sparse projects can be very dull. It can be hard to relate to “Hello World”.

To do this, you may naturally make the project more realistic. This is especially true if you are ADHD (like me) and you can only concentrate for long periods of time, on stuff that is at least mildly interesting.

The strategy here is to go ahead and make it interesting if you must, but handicap yourself, attitude wise. Explain to yourself that you’re taking some chances by adding breadth to the project that might be a bit distracting.

Many times, it turned out that I spent only half hour on learning xyz which was the point of an exercise, and a many times that on ancillary issues that were really only designed to make the project meaningful, such as realistic data inputs, or downstream distribution that felt meaningful.

An example would be “This project would feel much more real to me if it was like our other projects, which always FTP the results to the abc server.”

So instead of spending your time budget learning the syntax/semantics for xyz, you are configuring a dev instance of abc server, setting up permissions, etc etc.

How Not To Be Sidelined By Version Issues

Everywhere you look on the Camel site, you will see these odd references to what version which feature is deployed into. These are much more prominent than what you might see on other open source projects. Why such a prominent display?

Turns out that there is a very real reason, and I burned up some time trying to diagnose odd errors that were only caused by version differences. How could this be? Just use the latest, right? Not so fast …

The Camel codebase moves much faster than, for example:

  • Your deployment options. You may have to deploy to a much older version of Camel or Fuse or Karaf or EAP at work.
  • Your testing options. If you study on the latest, understand that the Red Hat Exam which costs hundreds of dollars might be written on a version of the API that doesn’t have lots of the features you are learning!
  • Tutorials, docs, sample code versions. Lots of docs, tutorials, and sample projects were written years ago. Helpful to have, except whoops Camel has moved on so what you are reading may not be relevant.
  • Your tooling options: Red Hat’s IDE comes to mind.

Main point?

  • Camel DOES NOT MIX WELL with other versions of itself. Don’t expect to use one camel version of one dependency, and another camel version of another dependency. You have to move them all in lockstep. You don’t want to find out the hard way what happens if you try to mix them.

Knowing what you are up against helps you plan your learning around these obstacles.

Workspace Configuration

From above, I assume you have

  1. An IDE such as a version of Eclipse
  2. The camel code.
  3. book code [optional]
  4. Multiple workspaces, as detailed below.

There are important workspace segregation strategies that I find helpful when learning Camel, one is a life-saver.

  • Keep your the Camel and Camel In Action codebases out of your working workspace. It’s too much code, and will suck up all your IDE’s memory before it even fully opens. Especially, give the Camel code it’s own workspace. Even then you’ll probably have to give your IDE extra memory.
  • Maintain a separate workspace for Camel code only. Build with maven so you have fully compiled code in view. Only import Camel modules that are relevant. Only keep open those camel modules that you really need to search in (usually only camel-core or the camel-xyz component that you are learning.)
  • At the first moment of cognitive overwhelm on any xyz topic, reflexively  grep for .json(in *Test.java” files first, where json is the xyz that you are attempting to learn. This one tip may save you quadzillions of hours. Search in both the book code, and in camel’s own source code. I use Eclipse file search (control H) for this grep, your approach may differ.
  • Be ready to use grep in the bash shell or comparable search to above in your favorite text editor such as sublime, for times when opening the IDE in the Camel workspace isn’t worth the trouble.
  • If you opt to maintain the Camel In Action book’s code as an additional resource – recommended – the same rules apply for it as for the camel code above. Separate workspace, fully compiled, grep as above…

Fan-Out is Actually a Different XYZ

I saved this for last, because it might be a bit of a head scratcher at first.

One thing I run into a lot has to do with fan-out.

  1. I am learning xyz
  2. I got it down, I think – my sparse project is working
  3. Feeling confident, I move on a new xyz
  4. Two days later, I discover that I didn’t have the previous one down like I thought… WTF?

The simplest use case I can think of is json. But you can apply this to many similar situations.

I found that going back and forth between json and other inputs/outputs was super easy, and fast to learn. But I only did so with single inputs, as per the rule above, of using sparse example code. Mission accomplished!

But the first time I actually tried to consume some real json, however, I try to injest a file with multiple json records, and I’m lost. For this, I had to learn to incorporate jsonpath, and splitter, and aggregation. Which in turn required a bit of research on the actual expression for jsonpath that had nothing to do with Camel. Hmmm.

So I had learned json marshalling and unmarshalling, but in another respect I really hadn’t. Because the fan-out to multiple records is a totally separate problem space. Welcome to your brave new world.

Fast Supporting Setups

Docker isn’t the only way, but however you achieve fast setups for your supporting infrastructure, this can be a good thing.

Here’s how I did mine..

When learning other skillsets this kind of shortcut might not be necessary. With Camel, every 20 hours you save might make the difference between giving up before you finish, or not.

Conclusion

Learning Apache Camel can take a long time, but there are strategies that can help you shortcut the process.

This article diagnosed a problem. To review one solution, click here.

Jammazwan – Projects for Learning Apache Camel

appwriter_logo

Camel Keeper: Jammazwan

“Jammazwan” is Hindi for “camel keeper”.

Apache Camel is sponsored and maintained by Red Hat, and a part of it’s Fuse offering.

Me, a jammazwan? another Camel Keeper: TL DR?     about:
pete jammazwanPhotoSmall jamzVid1

If you are impatient or in a hurry, the 2 minute video might get you through faster than reading this blog.

Ease Learning by Driving Adoption

Start with Camel projects that already do something useful. Projects that already work.  Every example project is designed to be IDE ready in 60 seconds. Try one!

Jammazwan lets you learn Camel the same way you probably already learned everything else you work with every day, like Maven or Spring or JEE or Eclipse or IntelliJ or…  Someone gave you projects that already worked, and you gradually expanded your skillset until you became an expert.

Make the Easy Stuff Easy

Once you succeed with Camel, you’ll want to use it for everything that’s easy.  That’s what Camel is about, making things easy, the stuff we have to do every day. And Jammazwan example projects are designed to be easy.

Let adoption be the driver for the learning that has to happen in your organization. Cryptic docs really start making sense once you’ve got projects that you care about, to give you context, and you’re not wrestling with syntax and semantics.

Why Jammazwan?

I wrote Jammazwan projects because my only option was to learn from books or docs or courses or the like.  Author Claus Isben, for example, writes a superb book. But you show me some working code that I start using in projects I need, and I’m off to the races. Only then do I become super-enthusiastic to use a great book or docs, to help me expand my project into a perfect use cases.

Jammazwan projects cover things that you need to do every day.

Then, Jammazwan.100 for Breadth and Depth

Let’s face it. Most of us work in pretty homogenous corporate environments. “We only use camel for …. type of use cases.”  Our skills then rarely expand beyond our daily use cases.

What I needed was a randomized project spec generator to push me past my normal comfort zone and develop Camel coding speed in areas that I don’t get to do every day.

Take a look at jammazwan.100 and jammazwan.100.done.

The areas you know will go fast, but Jammazwan 100 will also push you through unfamiliar areas, so you can get them up to speed, using the kind of reps you get in your daily work.

If You Must Study First

There are some things about learning Apache Camel that would be helpful to know. Before you invest a bunch of time.

Consult this whitepaper-like blog to save you time, by identifying some strategies and tips. Here’s hoping that it saves you a few days, or even weeks of time!

Other

If you are an automation enthusiast, you might look at the code in jammazwan.maker.

You might even want to use it to generate your own Camel projects. But it’s not documented or designed to be user friendly, so it may not be worth the effort.

Help?

Ping me if I can help  your shop move it’s development efforts forward with Camel.