Polymer/Lit Skills Update – June 1, 2018

Photo by Tim Mossholder on Unsplash

appwriter_logo

If you thought about hiring me to help you out remotely on your Polymer or Lit project, you’re probably wondering what kind of results you would get.

Update June 1:

This update is identical to my May 1 update, excepting this section.

The Polymer team announced their move to Lit from previous Polymer during May. This exposed a large gap in my skillset, which was still weak on the javascript side.

I spent most of my working hours in May doubling back on javascript and especially ES6 functionality. I still have a ways to go to catch up to the many pieces of javascript that are necessary to move to Lit, but May saw good progress in this direction.

Skills:

One way of knowing is to see what I’ve built with Polymer. It’s all open source, but that could be time consuming, as it involves over a dozen projects, most too lame to sift through. I’m very pleased with my progress, even though its less than a year’s aggregate work. Polymer is super-fun to work with.

The ultimate test of my skills could be viewed in my alpha app, deployed at pullmodel.com, but it’s still alpha and most of the app’s functionality is only visible at the higher permission levels. So I made this video to at least let you glimpse at the kind of functionality I can easily help you with. Or, this one from a couple years ago.

Limitations:

My strongest motive in this post is to accurately present my well earned humility with Polymer. Because I’ve seen what the truly great coders in Polymer know – I spend time with them almost every day on the Polymer Slack feed. I am constantly reminded how much there is to learn about this world.

I’m good at cranking out the work and solving problems, that I can offer confidently. Straightforward CRUD stuff, deployed on Firebase, you can engage me there and know that it will get done in a reasonable amount of time.

If you’re looking for the kind of blackbelt javascript work that will get you out of tight cracks created when your JS is too ambitious or advanced – I know those guys – but I’m not in that group. There are so many aspects of advanced JS that I know about, but have not fully conquered…yet.

My primary deficits are in the areas that I just haven’t needed so far, in my first apps. These include slots, direct interaction with shadow/shady/light dom via JS, writing promises and async behaviors, and anything beyond rudimentary CSS.

Last, if you’re wondering about Polymer 3, I have no experience so far. I won’t be taking this on until summer, at the earliest.

Summary:

Recapping from other places for emphasis here – I’ve got decades of experience as a dev, mostly in back end java – but less than 12 aggregate months of full time work in the javascript world as a front end developer – all on Polymer.

Rates:

Western economies seem to have different rates than the developing world, I’ve seen job postings that seem to imply $5 an hour, compared to Silicon Valley or Austin TX rates that climb ridiculously to the hundreds.

Given the wide variability in rates, especially for remote work, I’m pretty flexible for short term work – within the normal bounds of Western rates.

Need a Bigger Team?

Maybe you don’t want to just hire “some guy.” You need a whole team. I’m decently networked in the Austin market, and could help hook you up with any of several larger shops that could deploy a whole team, depending on your needs.

 

Polymer Skills Update – May 1 2018 – Pete Carapetyan

Photo by Tim Mossholder on Unsplash

appwriter_logo

If you thought about hiring me to help you out remotely on your Polymer project, you’re probably wondering what kind of results you would get.

Skills:

One way of knowing is to see what I’ve built with Polymer. It’s all open source, but that could be time consuming, as it involves over a dozen projects, most too lame to sift through. I’m very pleased with my progress, even though its less than a year’s aggregate work. Polymer is super-fun to work with.

The ultimate test of my skills could be viewed in my alpha app, deployed at pullmodel.com, but it’s still alpha and most of the app’s functionality is only visible at the higher permission levels. So I made this video to at least let you glimpse at the kind of functionality I can easily help you with. Or, this one from a couple years ago.

Limitations:

My strongest motive in this post is to accurately present my well earned humility with Polymer. Because I’ve seen what the truly great coders in Polymer know – I spend time with them almost every day on the Polymer Slack feed. I am constantly reminded how much there is to learn about this world.

I’m good at cranking out the work and solving problems, that I can offer confidently. Straightforward CRUD stuff, deployed on Firebase, you can engage me there and know that it will get done in a reasonable amount of time.

If you’re looking for the kind of blackbelt javascript work that will get you out of tight cracks created when your JS is too ambitious or advanced – I know those guys – but I’m not in that group. There are so many aspects of advanced JS that I know about, but have not fully conquered…yet.

My primary deficits are in the areas that I just haven’t needed so far, in my first apps. These include slots, direct interaction with shadow/shady/light dom via JS, writing promises and async behaviors, and anything beyond rudimentary CSS.

Last, if you’re wondering about Polymer 3, I have no experience so far. I won’t be taking this on until summer, at the earliest.

Summary:

Recapping from other places for emphasis here – I’ve got decades of experience as a dev, mostly in back end java – but less than 12 aggregate months of full time work in the javascript world as a front end developer – all on Polymer.

Rates:

Western economies seem to have different rates than the developing world, I’ve seen job postings that seem to imply $5 an hour, compared to Silicon Valley or Austin TX rates that climb ridiculously to the hundreds.

Given the wide variability in rates, especially for remote work, I’m pretty flexible for short term work – within the normal bounds of Western rates.

Need a Bigger Team?

Maybe you don’t want to just hire “some guy.” You need a whole team. I’m decently networked in the Austin market, and could help hook you up with any of several larger shops that could deploy a whole team, depending on your needs.

 

About pullModel

Photo by Thought Catalog on Unsplash

appwriter_logo

Drip Educate Me!

PullModel is for when you need information, but not all at once, please. 

Might not be as weird as you first think.

Video:

Every week I tend to add features and/or refactor, so this video is always out of date, but still it gives you a basic idea of the functionality.

#1 or #2? … No! #3

Information can generally fall into 2 categories,

  1. wanted
  2. unwanted ( this information showed up, and I never asked for it).

But what about a 3rd, weird category? Wanted information that is both sought after, but it shows up later, by design. That’s just crazy. Who would seek information – and then asked that it be delivered later?

Turns out, education fits this exact model. Stuff that we could never absorb in one big chunk. Then too, so do some subscriptions, such as newsletters that our financial advisor might send us.

In an increasingly over-informed world, drip education offers something intentional that might be counter-intuitive, information that comes later, by design.

Subscriber Motives:

You might start with two different motives for requesting that the information you want be delivered later, rather than now.

  1. TIME SPECIFIC:
    The information doesn’t exist yet, you want it when it on a schedule, as it becomes available. Again, your financial advisor’s newsletter comes to mind.
  2. OVERWHELM
    “I can’t even get my head around this yet.”This is why you take courses and pay a professor to spoon feed you over a semester, instead of just reading his textbook. First, you get this one main idea, then let it sit in the brain for n days. After that, the next idea will make sense. Lather, rinse, repeat, pretty soon you’re understanding complex ideas like E=mc2 that you never thought possible.

There are other motives, to be sure. PullModel doesn’t prescribe a motive, but it does offer a remedy.

Timed Information

pullModel is a timing engine for information. When you subscribe to a collection of information that you want to learn about, you get it on a specific time schedule, set by the author, from the start date that you subscribe to that information.

For example, if you subscribe to lessons from your running instructor:

  1. Day1: First information on the day you subscribe.
  2. Day7: Second information a week later.
  3. Day14: Third information another week after that.
  4. Day30: Fourth information a month after you subscribe.
  5. etc – any days as prescribed by the publisher of that information.

As an alternative (such as your financial advisor’s newsletter) pullModel allows you to send information to a subscriber, but on a schedule that you set.

Hopefully, that spacing/timing strategy works to your advantage. It’s all about you, the receiver of information, and what’s going to work best for your learning process.

What Medium:

Email? Text Message? Facebook Message?

Let’s just start with facts, again. We’re in an over-informed world, and it’s only getting worse, not better. Cutting off entire channels of information is often the only way to survive. “Don’t call me on my home phone, I never answer and don’t even check my messages very often!”

pullModel delivers information via email.

For those – and there are many – who never even check their email, but would still like to get their pullModel information, pullModel offers alternative notifications that an email has been sent to them.

  1. Text Message Notifications
  2. Facebook Messenger Notifications

These features are not currently turned on, but approximately 75% of the development has been completed on these features.

Ripe for Abuse:

pullModel, like all forms of information, is always ripe for abuse. Since it cannot change this fact, instead it offers the subscriber a remedy.

Here’s an example that lives outside of pullModel. I went to a wedding last night, and someone stood up to raise a toast, but then instead droned on for 15 minutes. Everyone except that person was ready to pull their hair out.

This is why pullModel is designed to make it difficult to become a publisher, and super easy to un-subscribe.

It is also why pullModel is provided as open source software. If you want to deploy it yourself, go for it. But you take the risk of abuse, not me. It’s your installation, your users.

Market Alternatives

There are an indeterminable number of great alternatives to pullModel for publishing your information in a drip format, some free, some easy, some powerful, some are super popular. They are usually variants on spamming systems for information providers such as vendors, or just newsletter software. They might include these great options:

  • AWeber
  • Mail Chimp
  • Constant Contact
  • Socket Labs
  • InfusionSoft

These are almost always better options than pullModel, because they are designed to be fully featured and typically heavily staffed. But there may be costs and/or technical capabilities required.

PullModel was created for a different purpose than email marketing.

Open Source

The code that runs pullModel is open sourced and may be used by anyone.

Future Plans

See Personal Motivations below, the plan for pullModel is to keep it in operation for a very small subset of people to use, and as a vehicle for my own personal motivations below.

As such, it’s maturity as a platform may always be questionable. The prospective user is always directed to the many alternatives above.

Primary Technical Features

All primary features are developed first in poly-on-fire projects separately one at a time. In this manner, you can isolate the code for achieving that feature without the confusion of the entire app.

These features are all pretty normal features as might be found in any modern application.

The poly-on-fire projects linked below are code building blocks or demo projects for this pullModel codebase.

  • cookies
  • query params
  • state management
  • CRUD: Create, Retrieve, Update, Delete
  • db permissions for actual data security
  • visibility hiding for functional UI security and workflow management
  • Authorization/Authentication using external OAuth (google, facebook, twitter, github)
  • messaging via SMS, Facebook Messenger
  • back end processing – fired by database events

 

 

 

 

The History of Polymer – Comic Version

Photo by Aman Shrivastava on Unsplash

appwriter_logo

DOM

In 2004 a committee produces something called the DOM, which was a browser spec.
It was thought of as inadequate, and there wasn’t much you could do.

Frameworks:

But wait! This is a chance to re-invent! Coders love that!
A succession of frameworks emerged, each competing to re-invent DOM better than the others, with bolt-on JS. There were many. Each, was of course, ‘the best’ especially compared to the other choices. Developers love that.

Some figured “Heh, since we’re re-inventing, we may as well throw in the kitchen sink.” So some of these frameworks got pretty fat. “We’ll run your app, but first it has to download 700k of minified js.”

Web Components, and Polymer

Years later, the chrome team somehow observed all this crazy activity and said “Hey, since everyone agrees the DOM sucks, why don’t we just fix it?” The web component spec emerged, not even sure by who, but google’s chrome team was all over it.

“Hey, this is a good spec, let’s implement it in our browser, and we can write polyfills for other browsers that aren’t ready yet. Then we can write a helper library to implement stuff like getters and setters, for this new spec to consume. We can call it Polymer. It will be more of anti-framework than a framework. Use the platform!”

The chrome team guessed that first Polymer would help, but eventually it almost would go away, as browsers implemented more and more of the spec as core browser functionality.

Frameworks Respond

“Oh great, now there’s no need for a re-invention of the DOM wheel any more!” said no authors of bolt-on frameworks, anywhere.

Even more new frameworks emerged, just in case developers needed more design approaches to evaluate, and more apps to be written in future abandon-ware.

Polymer Captures Massive Market Share

Many years and 2 major versions later, Polymer emerged as a prominent alternative to the other, fatter, do-more-better frameworks. Polymer captures over 1% of the front end market by January 2018, some believe!

Most developers refuse to mention Polymer as an alternative, because “It’s too hard, too low level, you have to know javascript”
Developers hate that.

Youtube and a few other google products were re-written in Polymer.

And that’s the end of Polymer’s short history. For now.

poly-on-fire: Polymer, on Firebase

appwriter_logo

poly-on-fire is my collection of commonly required functionality pieces, in isolation.

Intro:

The big gripe about Polymer – whenever I talk to front end devs here in Austin TX, is that it’s “… too low level, too hard to get things done, you have to know javascript.” I didn’t find that to be the case, maybe it’s because breaking the functionality I needed for my apps into these isolated, easy to learn pieces made it all work easily, for me.

Leaving them behind in github, as really dumb, easy, projects, makes it fast for me to re-learn them again when I forget. Or when I need some perspective. Especially compared to real apps, where everything gets all mixed up, and a little more time consuming to distinguish which component is doing what.

You’ll also find there, something more than only the most isolated pieces. Some projects then integrate more isolated pieces, in layers. One project for just CRUD, one for social logins, one for redux, but then an integration project that combines the CRUD with redux and social logins into a visibility controller, but without the complexity of a finished app.

Last but not least, it’s all combined into a more finished (never truly finished) app called pullmodel, which brings it all together and tests whether these pieces are ready for prime time, in a consumer oriented PWA for both mobile and desktop.

Find it all on github/poly-on-fire

The on-fire part:

There is no pretending, with poly-on-fire, in the sense that you don’t have to wonder how it would actually deploy in real life, with real hosting, on a real phone or desktop with real integration with a db and server side functions.

Because everything deploys immediately onto firebase, and firebase is free for modest usage, you can experiment to your heart’s content with the real deal. The only thing that changes for me when I go into production is that the increased volume will eventually start dinging my credit card.

I won’t even go into how cool this is compared to everything I’ve been through previously, at least not in this blog. Another blog, however, does make the comparison.

Logic:

The good news: Polymer isn’t a framework in the swiss-army-knife sense that most frameworks mean it.

The bad news: Polymer requires you to build your app piece at a time, with web components.

poly-on-fire is a collection of code repositories that explore how certain pieces might be combined or used together, for assembly into a larger app.

Use Cases:

Business requires more common features than is provided for by the Polymer Starter Kit.

If the feature is common enough to be needed in most any typical business app that I would have to build, but is not provided by the Polymer Starter Kit, this is a typical use case for a poly-on-fire project.

This lets me work out the basic kinks in a sample project, and easily migrate it to a real project when the time comes.

Such as:

  • How to bring in and consume query params?
  • Using Polymer and Redux for state management
  • Using Redux for visibility control (client side form of workflow and/or security)
  • NodeJS for server admin
  • SpringBoot for Firebase DB Admin
  • others

Polymer on Firebase

Polymer is a very small framework for building web components, presumably even into entire applications.

Firebase is an aggregation of hosting and 20 or so other products for deploying and supporting web applications.

Both are fully supported by Google and not necessarily intended to be used together, nor does it appear to be the common use case. But it can be a very keen strategy, for many reasons not clarified here.

Polymer Version 1,2, or 3?

All projects are currently written at Polymer 2 level.

There is an intent to migrate everything to Polymer 3 when it is released, but both schedule and my current lack of Polymer 3 skills mean this probably won’t happen anytime soon.

The Advantage of Entry Level Javascript Skills:

There are three very difficult aspects to learning Polymer, all of which are addressed by poly-on-fire projects.

  1. Polymer requires much more ‘low level’ JS skills than other platforms. Intentionally.
  2. Polymer enthusiasts tend towards advanced, and often don’t even remember what beginning JS coders have to learn.
  3. Poly-on-fire are isolated, fully deployable projects – including hosting code.

I started writing Polymer with no understanding of how javascript, the DOM, or the web component standard even worked. That’s pretty dumb to do, especially for a lifetime coder, but it did give me the chance to learn everything from scratch.

As a consequence, you may find that these projects are a bit easier to learn from, if you’re a coder coming from a non-javascript background.

You can deploy them for free on Firebase, and to a great degree, they should just work.

The Dis-Advantage of Entry Level Javascript Skills:

The veteran JS coder might find some of these projects too fundamental or uninteresting.

I have made an honest effort to learn everything I need to learn before publishing each example project. But that’s a fancy way of saying there may be subtle tricks that a 10 year JavaScript veteran could probably show me, and these projects might wish some of them to be included.

Please feel free to submit pull requests or even just a note to pete@datafundamentals.com, if you see anything missing.

Firebase Assumed:

For decades, we have avoided vertical stacks because lock in can be expensive and technologically limiting.

Firebase is free for low usage, and has excellent enough tech stacks to default to this platform first. It’s weird to hardwire to one vertical stack, but for the specific use-cases of these projects, it’s actually pretty appropriate.

Specifically, these firebase products are used in poly-on-fire, to date.

  • Hosting
  • Realtime Database
  • Cloud Functions
  • Authentication

PullModel

A more fully featured app with most of the other projects incorporated within is featured in pullModel.

This is a kind of seedbed for learning Polymer, for me. In it, you may discover that Polymer is a pretty powerful app.

With the exception of the workflow model – which took me foreeeeeeever to design and implement, even though it really isn’t very hard or even very much code – pullmodel proved to me that Polymer and Firebase make a pretty dang easy way to build a real PWA.

There are many aspects to PullModel, most noticeably service worker and testing, which are completely ignored to date.

Pullmodel is currently, and will probably always be, an incomplete, work-in-progress.

Autopm is the server side of pullModel, and does event based admin, responding to Realtime Database events.

Un-suck-alypse Ends the I.T. Boom?

Photo by Gianluca Zuccarelli on Unsplash

appwriter_logo

If IT was easy

If it suddenly became easy to write and deploy scalable, high quality software, the effects on the job market – and maybe even the real estate market – could be apocalyptic. Not that anyone would ever fear such an eventuality – the process of writing software has always kind of sucked. It’s not an easy process.

Software takes large teams and a long time. Deal with it.

I’ll spend the middle of this article rewinding why software is hard. Or, just skip to the bottom if you want the dreadful news, starting at the section 2018: Polymer and Firebase

Defining ‘Sucks’ in Software Development:

Here’s what doesn’t happen. Joe Bob or Mary Sue sits down and writes software. When he finishes, he sends you the link to it, deployed and running, then asks “What’s next?” Nope, that isn’t how things work.

Instead, there are so many things that just suck about the process, that it takes a whole team. Not just testers, but designers, a manager or two, database guys, deployment engineers, on and on. Teams evolved like this over the decades because stuff didn’t work as hoped “… oh, that sucks …” so the remedy is always to pile on more humans.

It might suck before you add the new team members, but eventually it works. Great apps are built. We have Gmail or Facebook or … pick your favorite app. Life is good.

Mary Sue is still working on that same app that she deployed months ago, but heh! There were some things that still sucked when version x was deployed, so we’re adding those features now.

The IT Marketplace

Mary Sue might have made $7 an hour in Austin TX as a hamburger flipper, but now that she’s a [insert IT role here] her benefits package alone probably costs more than that. With 10 years of experience, she drags down north of $50 an hour, plus benefits. Don’t even mention that these can be some pretty cushy hours, on some days.

When the product manager – yes, there’s one of those too – sits down to write the budget for the year – these budgets are in the millions. That’s what it costs, to write software.

Network Dynamics

A new feature is needed. Mary Sue might add that feature in a day, wasn’t that hard. But on a large team, the network of humans might multiply that cost by many times.

Product manager has to document it, that takes meetings with the designer and customer stand-in. Then the tester has to figure out how we’re going to test it. We have more meetings on schedule, more meetings on testing.

By the time this new feature is added on a typical team, well, you just have to budget some serious cashola, not just the day that it takes Mary Sue to add that feature. Meetings…

What if IT didn’t Suck?

Mary Sue sits down to add her feature, it takes less than a day because she wrote with reusable components that are easily available. Deployment? It just happens, it’s already scalable and super secure. The components are already hardened from years of testing, so hey, that new feature is done. Next?

The components and systems Mary Sue consumed were mature and standardized. Like Linux, there will be upgrades, but also like Linux, pretty much everything still works for years to come, so there’s no rewrite every 3 years to fit the latest fad UI platform. Your new app really is ‘done’ when Mary Sue deploys it, as opposed to the current economics, where the cost of writing the app is only 10% of it’s true cost where maintenance is included.

The problem? That team! You just don’t need it. Take a hot spot like Austin or Seattle, this could have a pretty devastating economic effects. It could even hurt Mary Sue, because now all of a sudden there are so many of her, unlike the current shortages we have become accustomed to.

But IT Does Suck! So Don’t Worry!

For every problem we solve, another three emerge. I could go over the long lists of solved problems from previous decades, but this would be boring.

Solved problems only increase our expectations which adds more cool features to the backlog. Joe Bob will always have plenty of work to do. App Servers used to be a big challenge, now you deploy on anything using microservices, so instead, large teams focus on the deployment challenges. Life goes on.

2018: Polymer and Firebase

Bear with me here, because Polymer deployed to Firebase is just an example of a wider trend. I’ll speak of them concretely, but, I’m guessing you can accomplish the same thing with their competitors.

I sat down in January to write an app, and just wrote it. Couple months. Lots of features, no team. Pretty decent security footprint, pretty impressive back end and front end, scalable beyond anything that I am accustomed to writing, runs great on mobile and any normal browser platform.

Heck of it is – I started Polymer as a near-idiot at javascript, so probably 1/2 the time was spent just learning. One can only imagine how easy the next apps will go.

Because it’s Polymer, I was just able to pull down web components out of the public domain, and I really like these components. Decent, usable stuff. Plenty of choices and configuration options. And I am totally impressed with the underlying foundations, this is all pure javascript/HTML/CSS no heavy, fancy frameworks that may be abandon-ware 2 years from now.

With Firebase, deployment, database, scaling, security, all this is pretty much taken care of for me. Thus far, it’s all been free – as in zero cash cost – but eventually I’ll pay on a usage basis when my app goes into production. Even then, I won’t have to worry about $ ever, because I’ll never pay more than the app’s usage merits.

My point isn’t perfection – these are still early days and I’m sure you could poke plenty of holes in my app, and the underlying systems. The point is – this is a huge shift. My skills might still suck a bit, but this process really didn’t suck.

Un-suck-alypse

Help me out here by repeating my thought experiment in your own way. What terminology would you use?

If it sucked before, and it doesn’t suck now, then that’s kind of an un-suck. Does that work? So then I suffix an economic catastrophe -where Joe Bob and Mary Sue don’t need the rest of their team any more, so there’s apocalypse.  Seems a bit extreme, but makes the point. unsuckalypse.

I’m in my 60s and I’ve had this happen to me before, when I worked in other industries and capacities. Entire workforces disappearing almost overnight. So for me this is a familiar feeling. It happens. Doesn’t mean it’s going to happen here, just a familiar feeling, that’s all. 1973. 1986. 2001. 2008. Been there, done that.

The Un-Suck-Alypse Might Not Suck At All

Bad situations can also be good. Gawd. Sitting down and writing a high grade app in a couple months! Wow, that was a cool experience. I want to do it again! Or even just add more cool stuff to this one.

So many nifty possibilities. What happens when millions of humans all over the world can simply sit down and write cool apps? Can’t even ponder of all the great consequences that might happen as a result. Can you? Mind bending.

What happens when it doesn’t cost millions and millions to under-write such an app? Oh my. Those are completely different economics than we have been working under, for the past 50 years.

Extrapolate that, a couple decades forward! Now there is a thought experiment for you.

Jenkins Pipeline TTFHW

Photo by Samuel Zeller on Unsplash

appwriter_logo

CI Server: No More Manual Project Setup:

 

One of the greatest ironies of recent times has been that the least automated part of our process has been the very heart of automation, the CI server. To set up each project, you had to operate a UI manually, only to have to repeat the process again manually if anything ever happened to that server, or even just that project. Crazy.

Finally, with Jenkins Pipeline – now every CI project can be code, and it’s stored with the rest of the code in the project’s SCM, where it belongs. Niiiiice. Other CI systems, like TravisCI might have arrived there first, but so many of us already use Jenkins, and the price is right, too.

Extra Win for Personal Projects or MicroServices

Jenkins Pipeline is even a bigger win for personal projects or microservices, because the proliferation of lots of projects on less stable machines is more common.

As an architect or product guy, for example, I might experiment with dozens of projects over a period of weeks, trying out different combinations of platforms or tooling, to get just the right combination. Using a CI server for such projects would be extremely time consuming if I had to click through all those installs with a UI screen.  And I wouldn’t even use a cloud instance for such a server, I’d grab some abandoned local box and use that. But only if it was fast, and the setup was disposable.

13 minute Walk Through

It took me longer to slog through the docs on the Jenkins Pipeline than it should have. The docs are good, and thorough. I just needed a faster ramp up. And a much smaller set of options, I only care about pipeline code that lives in my source repo.

So I built my own faster ramp up tutorial. If you watch the youtube at normal speed it takes less than 13 minutes. After that, you should be able to repeat it very quickly, as I also included four accompanying TTFHW projects, which are designed to work together to get you up and running with Jenkins Pipeline right quick like.

Don’t even worry if you want to move it to another box or boxes later. That’s what Jenkins Pipeline is about – the server is just that – a server. Your pipeline code lives in the project source, where it belongs.

 

Bare Bones MicroServices Camel

Photo by Dakota Corbin on Unsplash

appwriter_logo

The purpose of this article is to help you get through some very quick and concise examples of microservices using Camel.
This might also be called a “bare bones” approach.

Before we start, I’ll try to clarify some potential confusion.
This may take more time than reviewing the code itself 🙁
Skip through, accordingly. The meat is in the bottom half.

Micro Service, Maximum Confusion:

Microservices are best characterized by what they are not. Which leads to confusion, because a microservice which is not anything, also fails to exist. So it has to be something. Stay with me here.

As soon as a microservice becomes something, the arguments begin. “I’ll use REST,” so someone blogs “Microservices suck because REST sucks.” Then you have to explain to the hater that microservices don’t require REST, which may only confuse him, because most, in fact, may use REST APIs. And so on, through every decision point. Exhausting.

It was tempting to name this blog “Haters Guide to Camel Microservices” but that would only serve my own frustrations with the human side of this puzzle.

Microservices Defined In This Context

In this context, I will approach microservices as an attempt to eliminate the ball and chain of required infrastructure. No hosting Tomcat or service bus … or …. whatever might be getting your dander up. It can be a long list. Every technology has it’s own haters. Go to any conference. Embarrassing, at least for the technology agnostic.

The approach here will be to eliminate everything possible – the bare bones approach – because that makes Camel microservices easiest to learn.  This is actually a big win, because it is easier to learn how to add stuff back in, than learn what can be taken away after it is added. Or, so it was for me.

Next, a Spring Boot project will make up the second service. By adding the absolute minimum Camel code to this, you will quickly see the delta. Not much to see.

Debating Society Disclaimer

Given the impressive debating society around all things microservices… Please do not use these two projects as part of this debate.  These example projects are learning tools only, and have no merit, other than a place to pick up how to information quickly.

Why So Difficult To Find a Bare Bones Example?

The last piece of confusion to resolve is this:
If Camel requires nothing more than a bare bones setup, why is there almost no prominent online example of same? Excepting for unit test projects, which might only confuse the novice? Even Claus Isben’s own Spring Boot Example seems to include layers of opinionated infrastructure, which can obfuscate how bare bones a Camel service might be.

You won’t have to think hard to answer this question. No one really believes you would ever want to deploy a bare bones service. “You’re at least going to need this and this and this.” Great for teaching this and this and this, not so great for learning the essential minimum.

About the Routes

All routes in this set of examples try to do as little as possible, other than prove that they are running.

It is assumed that you would  or have learned about Camel routes elsewhere, and would insert your desired routes herein. One such modification would be to provide a serviceable camel endpoint such as REST or file or jms or  … which neither of these services has 🙂

Two of Quadzillion Deployment Methods Provided For:

So many options! But not here. I run both projects with either of

  1. Shell script (java -jar etc)
  2. Dockerfile

What is not included? It is assumed that the reader either has his favorite microservice vendor – i.e. AWS – or he is being assaulted daily by many vendors who would like to be his favorite. Nearly any could such platform could be consumed, with 1 or 2 above, and a little extra account metadata.

Each project renames the jar which maven produces to make it easier to point the shell script or Dockerfile. This is done using <finalname> in the pom.xml.


Series Advertisement: Jammazwan

“Jammazwan” is Hindi for “camel keeper”.
This blog is part of that series.

Me, a jammazwan? another Camel Keeper: TL DR?     about:
pete jammazwanPhotoSmall jamzVid1

If you are impatient or in a hurry, the 2 minute video might explain Jammazwan in the shortest amount of time.


Bare Bones Camel MicroService:

Github: BareBonesCamelMicroService

Little more than a main and a route. Just to show you how skinny it can get. Too skinny to be representative of a real service.

myMainClass

This is pretty much the only code in the project. Of these 24 lines, only 3 make this a Camel microservice:

Main main = new Main();
main.addRouteBuilder(new MyRoutes());
main.run();

The rest of this code is just imports, a main(), and the RouteBuilder with a 1 line route, which you would expect in anything Camel, microservice or not.

Note that I also had to add the maven-shade-plugin to the pom.
I left some sample shell scripts to launch it as either a  straight jar, or a docker image. Adjust scripts as needed to match your own file placements.

Other than that, the project is nearly empty. And now you know how to create the skinniest microservice imaginable.

Note also that no port is exposed in the Dockerfile, as this service has no endpoints.

Bare Bones, except adding Spring Boot

Github: SpringBootBareBonesCamelMicroService

What would be the absolute minimum number of steps required to use Spring Boot as it comes right out of the box?
This project answers that question.

Here’s a screenshot of how I generated it. Spring makes start.spring.io useable for anyone to use:

startspringboot

So far, no camel code whatsoever, other than naming a dependency in the pom.xml (lower right). A 30 second operation, start to finish.

Look below, and you’ll see exactly what was added after downloading. Notice that the three highlighted lines are literally same as those in the BareBones example above. The route is the same, and the primary difference is the different wrapper to run those three lines from above, only in the boot application.

addToSpringBoot

As with the previous project, the jar name was adjusted in the pom, and shell scripts and Dockerfile were added for deployment/running.

I’m hoping by now you would agree, there isn’t that much work to a Camel microservice, if you want to go bare bones. And repeating from above, any cloud vendor would be happy to take it from there. You can repeat this entire process in less time than it takes to read this blog, once the process is familiar.

Now That You Have Learned The Minimum:

You may now add back in whichever pieces of infrastructure technology that don’t get your dander up. I’m sure you have your own favorite list.

Most will make the right choice, and head towards something like Claus Isben’s adaptation of Spring Boot. If you do that now, you would at least be well informed. It’s not quite as well documented, but now you know the essential minimums.

Or, better yet, take full advantage of fabric8 and/or Openshift!

Go for it!

Be Glad I Didn’t Write Apache Camel

Photo by Josh Calabrese on Unsplash

appwriter_logo
If I had written Apache Camel, instead of my personal heroes – James Strachan, and Claus Isben – I would have completely screwed it up.

A nightmare. Allow me to clarify:

Explicit Everything – a Religious Belief:

In my world, the very word “implicit” is synonymous with Evil.

Implicit languages, implicit conversations, implicit meanings – these are all destined to confuse and sometimes even disrespect people. Or at least, disrespect their time.

Everything implicit, to me, is like an “in group” where only the initiates know the secret language. You can join the group, but only after you learn some implicit meanings. To heck with that.

Implicit languages, for example, are a little faster to code in, harder to tool, and much harder to learn than explicit languages. Or at least, that’s my perspective. Java is dumb as a box of rocks, compared to JavaScript – and that’s why I like it better. A reasonable example that favors the explicit.

I could go on and on, but it would only be more of the same. Law, governance, politics, culture, implicit vs explicit is a key issue everywhere, and a vast source of mischief.

Camel Works. Because of Generics. Implicit everything…

My worst nightmare is projects like Camel, where I cannot use explicit hierarchical topologies to walk a tree of types or functional options.

Camel uses generics, casting, and “type converters” to allow what amounts to an “any to any to any” kind of message passing. All without your explicit knowledge. If implicit was evil, this is evil on steroids.

But instead of being evil, it just works! Implicit is what allows the vast, and growing, array of Camel components and other usages to just work. It may make me appear lazy, (I am), but the extra work of testing at runtime really isn’t that bad, after making the mental adjustment.

Topologies as a Police State

Compare that to my chosen approach: Everything is a finite, known, hierarchical topology.

Except who makes the topology? Is it me, you, or ??? Now, we have a classic case of the Police State to the rescue.  I would probably have installed myself as the policeman, and Camel would never have grown beyond my ability to police the topology of types and functions. That would be amount to 1% of it’s current scope.

But that’s OK, because it would never have grown anyway.  Nobody even likes participating, where a police state is involved.

Conclusion

I have ragged on and on about Apache Camel and how needlessly hard it is to learn. I am not completely off, in this assessment.

But the point is that there is some magic to this design. It’s difficulty is in it’s brilliance. And it isn’t even that hard, you just have to get used to testing well at the runtime level.

No. That isn’t so hard.

 

Camel CXF – X__project Supplement

appwriter_logo

Who:

This blog should be interesting only to someone actively attempting to get their first camel-cxf project working. It is a companion piece to a progression of 4 example projects, and the docs on each.

  1. Code first CXF – simple.
  2. WSDL first CXF – simple.
  3. Code first CXF – complex objects.
  4. WSDL first CXF – complex objects.

It is notable that none of the previous Camel example projects required anything close to this kind of documentation, or explanation.

Main Points:

If you are impatient, like me, you may find that getting to number 4, above, is needlessly difficult, the first time. Especially aggravating, given that number 4 is the only variant that commercial/corporate applications might accept.

Then, when you get it all done, you might wonder what the fuss was about. Maybe this blog and these projects will get you there without spending much time.

The idea is to be able to blast through each of these very quickly, picking up additional steps/dependencies with each one.

The Sequence:

  1. Code first simple – basic dependencies, server, testing, tooling.
  2. WSDL first simple – add WSDL first steps, code.
  3. Code first complex – adds objects for input and output, processing, and testing variants.
  4. WSDL first complex – adds a merge with your own entity code.

Again, once you know the above steps you’ll wonder what all the fuss was about. Until then I found lots of ways to guess wrong and waste lots of time.

Specific to WSDL-First:

If you are new to SOAP and WSDL first in general, there may be some confusion about the generated code step. Since there may be some options, here is what I found successful.

  • Just running mvn compile is enough to generate the wsdl2java step. It’s will place it wherever you told it to in your pom, or in this case target/generated/src/main/java.
  • Wasn’t sure if I should be making this a legit source folder? Or copy the code to my src/main/java? I chose the latter, see next step.
  • When you copy it to your source folder, there still may be plenty of changes you have to make by hand. For example, your complex input and output object entities may already be in your codebase, and the wsdl2java generates you new entities. So in that case, you would have to merge the generated into the previous entities and then modify the rest of the generated code to point to those, and not the generated. You then delete the generated.
  • You may also have to comment out your wsdl2java plugin code in the pom.xml as soon as you have copied over and modified any generated code. This may or may not be obvious, but will make itself quite obvious if it becomes an issue.

Consuming WSDL from CodeFirst

One possible source of confusion can be quickly clarified by acknowledging a cheat that was used on the two WsdlFirst projects.

One of the most significant challenges to a WSDL first project of any type is the creation of the initial WSDL. This file is a complex 5 part file with many cross-referencing parts. Not for sissies like me, putting together silly example projects.

What was done on these example projects was to create the WSDL using the code first project, which creates a wsdl and publishes it to the browser as part of it’s functionality set.

This WSDL was then hand copied to the correct folder in the WsdlFirst projects.

A side benefit was that the WsdlFirstComplex and CodeFirstComplex projects could then also share entity classes, but that is just a side effect. In this case, the shared entities were placed in the jammazwan.shared project. Both WsdlFirst and CodeFirst projects share the same test code, using this approach.

Running Your Test as a Server (WTH?)

Problem: Jammazwan example projects are for learning Camel, not production deployments. Server and deployment code is intentionally excluded so you can isolate only what you are learning. So what happens in a case like this, where we actually need a running server to test with during development?

Solution: A quick, and ugly, manual hack is provided in our test code. I cringe, but it works great for the learning process.

A more realistic solution begins to outline itself in my Camel Microservices blog, though that is a bit off topic for this article.

Sequence:

  • Write your project to test with a normal test, such as template.requestBody(), like any other camel route.
  • Once it’s working, test it in the real world using something like SoapUI. To do this, uncomment the hack line

    Thread.sleep(Long.MAX_VALUE); Ugh.

  • You may now run the test, and the test basically hangs. Whola! Your server is operating. Even if you are wincing in disgust.
  • From your camel-ctx.xml file, copy the url. Such as http://localhost:9876/passThru/
  • Append ‘?wsdl‘ to this line. You now have a wsdl url of http://localhost:9876/passThru/?wsdl
  • Open the wsdl url in your browser. My chrome browser shows the xml from the wsdl file.
  • Now you can take this same wsdl url to your SoapUI tool, and test the actual working server with actual working xml. I won’t cover this here, as it’s assumed you already have this skillset.
  • Now you can kill your running test, which never finished, and comment back out the Thread.sleep(). Your manual dev test is complete.

Remaining Documentation:

Each of the 4 cxf example projects is documented in their respective NOTES.md

The NOTES.md files each focus on a diff between that, and previous artifacts. So 4 isn’t so much of a standalone, as a diff between it’s artifacts, and the previous. Etc. Other x__projects are standalone, not these 4.

  1. Code first CXF – simple –  NOTES.md
  2. WSDL first CXF – simple-  NOTES.md
  3. Code first CXF – complex objects –  NOTES.md
  4. WSDL first CXF – complex objects – NOTES.md