In this installment of our video podcast DevDen Open Office Hours, we discuss our migration to a new blogging platform and its implications at merge time, the Better Pull Request blog, some advice for handling conflicts for newbies to Git, and other exciting topics.
Grace Francisco: Hello and welcome to our next episode of the Dev Den Hangout. I'm Grace Francisco, one of your hosts for today. I am the head of developer advocacy at Atlassian. I have with me a team of experts today. I have Ian Buchanan out of Munich.
Ian Buchanan: Hello.
Grace: There is Ian, and Nicola Paolucci in Amsterdam.
Nicola Paolucci: Hello there.
Grace: Then Steve Smith, also in Amsterdam.
Steve Smith: Hello.
Grace: Then Tim Pettersen out of San Francisco.
Tim Pettersen: Hey, folks.
Grace: Today, we've got a whole bunch of great topics to cover, starting with Nicola. Nicola, tell us about the new blogging process that we're following.
Nicola: I find this very interesting to talk about and tell, and share this really cool thing we're doing. Which is, very simply, we have restructured and adopted a different model, different process to publish our blog. We actually have transitioned all our technical content, and the intention of writing technical content in the future, to a new blog at developer.atlassian.com/blog that has a completely new technology stack and is much more developer-friendly than our previous process.
Before we have, and we still do have a quite sizable and very performing, very advanced WordPress deployment for our corporate blog, which is blogs.atlassian.com. We wanted something a bit more tailored at developers, and where we can share our own hacks and our own findings in the tech sphere.
We created our own basically, and what we picked was... we use a static site generator, which is a fancy thing to do nowadays. It is a very cool way where you can have a super performing blog system that shares only static files. The one that we chose after a review as being based on nodeJS, and it's called lineman.
Obviously, to store actually the sources of our articles, we use Bitbucket. We're big fans of simple static text files for our articles, so we use markdown. The only thing that we needed to track some dynamic content and interaction with our readers is a way to track comments. For those, we use Disqus.
More than the tech stack, what they wanted to study is our process. Obviously we use Git and pull requests to manage and publish our blog. The way we work is this. Any branch that gets merged to master goes directly to production. Each branch might contain one article.
The next stage of our blog, so the next post or the next few posts that's going
to go live are branch off develop which is, let's say, work in progress branch.
When I want to write a new article, I'll create a new branch off develop. I name
it with a special prefix which is blog so
blog/ then the topic or the article
name. There, I write my own markdown files. When I'm ready to share this work in
progress with my colleagues, I open pull request. We have a review that can
happen very nicely on Bitbucket. We have a live preview of what's happening on
our staging site. We talked more about this a bit later.
After the process works out and we fixed what we needed to fix and everybody is happy about it, we generally want to add at least two approvals for each article. Then, we go through also a final review at the end. After that, then we merge it to master and then we go live. That's very roughly, very quickly the process.
One interesting bit that came up in developing this solution was how we show on staging more than just one article at a time. Initially, we were like everything that was merged to develop was going to staging. Then, sometimes you want to change the data thing so you need to rework and move off branches, one article that's going to go live. We actually came up with a very clever way to do it. Tim wrote an article about it. Maybe, Tim, you want to explain a little bit this advance technique that we use for our staging server? I think people will find it very interesting.
Tim: Yeah. For sure, Nicola. As Nicola was saying, we have this amazing process now for staging our blogs and getting into production. The only problem is, because we built up this awesome process, all of the other developers at Atlassian are very keen to start working as well because it beats the hell out of WordPress, which is what we're using before.
One of the problems of having this increased uptake in office is that it means that multiple people developing their own blogs and trying to get them onto the staging server at the same time. This meant that we've had different individual feature branches as Nicola mentioned. Each blog is being developed on its own branch trying to be staged at the same time and we're basically stomping each other's changes.
In fact, I can probably give you a little demo to illustrate that. I'm just going to try and share my screen here. Can you see my terminal there?
Tim: Cool. Fantastic. What I'm going to do is run Lineman, which is the little static site generator that Nicola was talking about. We can actually see a local copy of our blog actually. That probably means I'm probably going to need to share more than just my terminal so give me one moment. I'll share the entire screen. There we go. The lineman's pretty quick to run up and we're just running a little local HTTP server, which is going to be serving our static content. And we're currently running from a branch called "A Better Pull Request," which is a blog that was published last Thursday.
If I go to my browser and hit
http://localhost:8000 you can see our developer
side. And if I go to the latest blog posts, you can see the latest blog post is
"A Better Pull Request," because we're running from this branch, which was
actually made to develop last Thursday, but it'll do well enough for these
demonstrative purposes. But if I change to a different branch...whoops. There
are a few too many branches.
Let's check out "Getting to Know io.js," which was another blog that was
published last Friday and then refresh our page. You'll see that the latest blog
post is "Getting to Know io.js." But that other blog post that we were just
looking at, "A Better Pull Request," has disappeared. If we flick back to "A
Better Pull Request" again...I'm going to use this cool little command,
checkout - If you haven't seen this before, it's pretty cool. It'll go back to
the last branch that you were working on. So it's a nice way to quickly switch
contexts if you need to be working on a different branch.
But if I go back and refresh, you'll see that "A Better Pull Request" is back at the top of the list. This is effectively what was happening on our staging server, because basically we'd have multiple people developing the same branch, developing their articles, pushing them to the staging server. Then our continuous integration and continuous deployment job would be deploying that latest branch on top of it.
What we did is we built a little tool that solves that problem. What we did is we considered the different solutions. There's a few obvious ones. The first is that we could have one staging server per branch. The problem is with that is it could get relatively expensive, because if you have 10 different branches on the go at the same time, you're going to end up with 10 different staging servers. The second solution that I came up with is I would just continually re-base my blog branch and keep pushing that to the server so my article is always deployed and stomping over the other authors. That, obviously, is not a great idea in a team context [laughs], and directly violates one of our main values, which is "Play as a team." The third solution we came up with was "Perhaps there's a better way we could solve this using Git." Now, Git has a bunch of different merge strategies. One of them is a very special merge strategy called the Octopus Merge that allows you to merge more than two merge heads together.
So we had this theory that we could potentially merge all of the different articles that were ready to be published or ready to be reviewed into the same hedge and then just deploy that merge result to the server instead of just deploying a single branch head. I might just share my screen again and I can give you a little example of what that looks like. Sorry, one moment. You can see my terminal again?
Tim: I've got a little empty repository. It's just got one commit in it. And
it's got this little Octopus-building script. Basically what this is going to do is
generate eight different branches, each with three different commits on it. So
if I do
build-octopus.sh, you can see that's generated a whole bunch of
different commits. We now have eight leg branches, obviously, because it's an
Octopus Merge. The way that we do an Octopus Merge is simply running git merge
and then typing in the names of the branches that we want to merge. So, it's
just like a regular merge. You just give it more than the additional head. Bear
with me for one second. And the other thing I'm going to do is use the merge
strategy. We're going to create a new merge commit. This will be the body of the
octopus, if you will.
That's created a merge commit with eight different parents, and just to
illustrate that I'll run
git log, and you can see we have this quite
attractive-looking Octopus thing going on with our merge commit at the top, and
then every single branch being merged into that. This is effectively what we
wanted to do with our blog site, because we wanted to have of the different
commits that people had been independently developing and then merge them all
The only thing, there were a couple of special constraints that meant we couldn't use an Octopus Merge per se. The first kind of constraint on us was that this merge, because it's going to be running on a Bamboo server, so basically running non-interactively, it can never, ever fail with conflicts. Because if it fails with conflicts, that means that it's not going to be able to deploy, because it will have this conflicted working copy, and that's not something that's appropriate to deploy to a staging server. That basically meant that our octopus merge would have to be run in such a way that it would never conflict.
The second thing that we needed to do is to make sure that the changes that end up in the deployment are only static changes, so actual blog content. Because that repository contains all of the actual site itself, so, all the logic for building that kind of attractive front page -- if any of that stuff changes, we don't want to automatically merge that, because there's a chance to get logical conflicts. We only actually want to deploy individual article content.
The third thing that we wanted to do is allow you to not stage your content if you didn't feel like it was ready for review. So, we didn't want to just be taking every single branch in the repository, bundling it all together, and then pushing it up to the server, because, potentially, there's some sensitive content, or, potentially, you're not ready for your things to be reviewed just yet.
What that meant is we built, instead of just using up the first merge and
putting that into our Bamboo script, we decided to build a new little tool
git merge-distinct is, basically, a wrapper
around an octopus merge.
I can probably just give you another quick demonstration on how that works. We just changed the screen once again. Do you see my terminal again?
Tim: Cool. OK. I'm going to pop back to that command line that we were
looking at before. I'm going to make my terminal a little bit bigger here. OK.
I'm going to
git checkout develop. Just a quick look at branches we've got
here. We've got
develop and then we've got two outstanding blogs, those two we
were looking at before --
What I'm going to do is run
git merge-distinct. The first thing I'm going to
do is, say that we only want to include branches that have changed paths
app/posts. What this is going to do is it's only going to allow
branches to be merged into this deployment artifact that have content that's
If the author of that branch just changed some content outside of that, so, possibly, some dynamic content, which is used as part of the core functionality of the site. Then that would be included in the actual branch that's going to be deployed.
The second thing I'm going to do is specify an additional qualifier specifying
which branches I want to be merged. In this case, it's going to be anything
names based under
blog/*. Basically, this means that only these two
articles are going to be deployed.
This other branch,
series/continuous-integration, which, because it doesn't
start with "blog", is going to be consider private, and not something which you
we want to deploy. When we run this command -- fingers crossed -- excellent! It
comes out with the output
Merged 3 parents.
Basically, it's merged
develop, which is the branch we were on originally, and
then these to blogs, which is exactly what we were hoping for. Now, because our
main server is running in the background, if we just pop back over to our local
host and hit "Refresh", you see we've got both "Getting to know io.js" and "A
better pull request" deployed at the same time, which was the intent of the
Nicola: Way to go.
Tim: Cool. [laughter]
Grace: Great. So, for the folks who were on the line right now -- we've got quite a number of viewers -- feel free to use that Q&A to ask us questions at any time. On to Ian Buchanan, who's at OOP 2015, in Munich. Tell us, Ian, how are things going out there.
Ian: Sorry for that, couldn't find the mute button. Yes. Well, there's [laughs] enough snow here to make me sympathetic to those in New England. We're seeing some snow here too. Probably not as much. We're looking for to having Sven talking about Atlassian coding culture tomorrow, and I'll be talking about the business value of git on Thursday. So, if you're in the area, pop by and see us. I spent the Monday tutorial building a domain-specific modeling language. That was really fun. This morning, I've listened to a talk about agile fluency and just saw a talk about sociocracy, both very interesting things. Coming up are some additional talks about automated testing, continuous integration, continuous deployment, and all things agile.
In any case, if you're interested in any of those topics, I'll be blogging about some of those quite soon. At the booth, I've been having some fun discussions about how Bamboo is better than Jenkins, or how Bamboo can be used with Android, both some things I've been looking into lately.
Anyway, if you're listening from Munich, drop by the booth and have a chat. That's pretty much what's going on here.
Grace: What's the most popular question you're getting right now, over there?
Ian: It's really a class of questions about Bamboo. I think that's mostly because other folks at the booth feel like Bamboo's very technical and they're, "OK, all those questions go to Ian."
Ian: But among them are things like what Bamboo is, how it works, and where it fits into the develop lifecycle. Mostly, I think, if people get interested in it, they know what continuous integration is but are just really looking for product differences.
Grace: Super. Well, make sure to tackle Ian with your toughest questions at the OOP Conference, if you are there. [laughs] Thanks, Ian. Steve, you're going to be coming out to San Francisco soon for DeveloperWeek. Tell us about that.
Steve: Yeah. Hi, everyone. I'm going to be at DeveloperWeek in about two weeks' time. What I'm going to be giving is a tutorial/workshop on Docker. The background to this came about when Nicola, who you maybe will guess is sitting next to me, has been giving all the talks about Docker for developers. But I've noticed this when talking to developers, there's still a lot of confusion about exactly what Docker is and how you'd use it and why you would even want to use it, especially if you're a non-operations person. So what I'm in the process of putting together, and I'm actually testing these out with Nicola and some of the developers here in Amsterdam today, was a bit of a tutorial about the background Docker, about how it fits together, some of the end-of-line technologies about it, and then an example project that will use Docker to do some testing.
The project I'm going to be testing has some complexities around testing, which is to do with we have to use real Postgres, some real Elasticsearch, to do some real-time testing. What we do is we use all the technologies and the concepts we learn in the Docker background session to build up a set of Docker containers that we can use to test our tricky little application, and then we go through some more stages of simplifying it and using some of the higher-level tools like Fig to simplify it and eventually integrate it into a continuous deployment pipeline using Bamboo.
It's an hour session. There's quite a lot had been packed in there. Hopefully, we'll get through all of it. However, I will be hanging around the entire week of Developer Week. Hopefully, I'll be able to have a lot of conversations with anyone who's really interested in learning a little bit more about this.
This is exciting stuff. It's very new area and it's changing rapidly. I'm trying to, hopefully, introduce some people to some of the core concepts to help them sort of reason through the rapids early changing landscape of the Docker and container ecosystem.
There's also going to be Codeship with this. Code is already out there. I think, Grace, if you could probably paste the link to it which should be on our confluence page. There's code that you could sort of follow along with or if you just want to follow along afterwards or use the slide as a workbook, then hopefully that will be of use as well.
That's going to be on the ninth of February, I believe, on Monday. If you're at Developer Week, come along. I'll come along to session. Just come and see me and we'll have a chat.
Grace: Super. Thank you, Steve. Nicola just posted a link for that. Thank you, Nicola. All right. Moving right along. Tim, you had an interesting blog post on Bitbucket pull request last week. You want to talk about that a bit?
Tim: Yeah, for sure. Absolutely. This is a kind of interesting post because it's a feature that's actually been in both Bitbucket and Stash since their conception so we're kind of talking about a few years. We realized recently that people may not realize that it's actually a feature. What I'm talking about here is how Bitbucket and Stash do their pull request diffs. I might just open the article so I can show you a couple of images from it. You just share my screen. When you think about a diff in terms of Git, quite often you're just thinking about the changes that have occurred on a particular branch.
Say you're doing a pull request the old way so back before you had Git, Bitbucket, GitHub, GitLab or Stash. You just pull the branch down from a server and you wanted to see what had changed.
Now, you might think that using this git diff triple dots syntax to see the changes on that branch is the reasonable way to do it. This does actually show you the changes that occurred on that particular branch. In terms of a pull request, which is actually not just a diff. It's actually someone's intent to merge some changes from one branch into another. This doesn't quite show the full picture.
The way we decided to implement pull request diffs in Stash and Bitbucket is to actually show you the difference between the two branch heads. The reason we do that is because if you just look at the changes that have occurred on the branch that you're merging into your target branch, it's actually going to be missing any changes that have occurred on that master branch since then.
You might be workingndering why this is a problem. I was actually explaining this to my wife when we're walking the dog yesterday. I was trying to think of a nice approachable way to explain why this diffing algorithm can be problematic or why you need to actually look at the changes on the master branch as well.
The analogy I came up with is imagine you have two different developers working on a cook book. Say you've got three recipes in that cook book. One of them is a recipe for some pizza dough. Then, that's already been sensibly completed. Then, these other two, well, chefs are writing recipes that will be contributed elsewhere in the book. Say maybe one is writing recipes for calzone and then the other one is writing a recipe for some pasta.
While they're working on their individual recipes and they're working on individual branches because they're using branch base development, one of the chefs realizes that the pizza dough recipe is missing salt in it and that's something obviously you have to have in your pizza dough. They go and update the pizza dough recipe and say, "Look. You need two tablespoons of Himalayan rock salt." Then, they commit that on their feature branch which contains this other recipe.
At the same time, this other developer...Sorry. This other chef -- I'll use the term chef -- is working on their own recipe. They also realize that that recipe is missing salt. On their separate feature branch, they make a similar change where they add two tablespoons of sea salt to the recipe. On each of their individual branches, the recipe is now complete. The recipe for pizza dough is now complete at least.
They've made slightly different changes. One of them said, "We need Himalayan rock salt." Then, another one said, "Hey, we need sea salt." Now, the problem comes when you merge these branches together. Because when they complete their own individual recipe and merge that back into master, it's going to add two tablespoons of salt to this pizza dough recipe.
The second chef also merges their own recipe into master. It's going to add an additional two tablespoons of salt to this pizza dough recipe. You're going to end up with four tablespoons of salt and a super salty pizza recipe.
That's why it's important to share the changes on master as part of the diff. The reason being, that when one of the chefs create...Well, when the second chef creates a pull request from their recipe branch back into master, they're going to see in the diff that there's already two tablespoons of salt there.
Now, it does seem kind of a maybe drawn out in that analogy. I highly recommend reading the article if you're curious about it. I'll just post that into the blog, sorry, into the chat window.
There is another reason why we show this type of diff as well. The way that we calculate this diff is actually by creating a fake merge commit in the background. There isn't any simple command you can run in git to see this kind of diff that shows you all of the changes on both branches. What we actually need to do is every time you update your feature branch or you update your master branch, both Stash and Bitbucket create a merge commit behind the scenes.
Then, they show you the difference between master and that merge commit, which is basically illustrating all of the changes that are going to happen on your master branch when you hit that merge button in the pull request. It gives you kind of the most accurate diff you can have because it's really showing you how your branch merge is going to affect master and affect the code that you're intending to ship your customers. That's really where we feel it's the best diff you can have.
The fact that we create this merge commit also gives us another advantage because it means we can detect ahead of time if your branch is going to conflict because obviously that merge commit is going to conflict. This is really handy because it means that the way we handle this is we actually commit all of those conflict markers that get generated by the Git merge command into that merge commit.
That means that when we show you the diff to the merge commit, we can actually show you if your pull request is going to conflict. That means that you can discuss with the other developers on the pull request how you're going to resolve these merge conflicts. Since merge conflicts by its very nature is obviously going to involve more than one developer, we think that the pull request is the best place to discuss how you can resolve these merge conflicts.
That's basically it. As I mentioned before, it's a feature that's been there since the beginning of time. Massive kudos to the Bitbucket and Stash teams for coming up with this merge algorithm. All I really did was kind of show people how we built something kind of awesome. [chuckles]
Grace: Your blog post created quite a bit quite of discussion on here. Nicola, you made some observation spots about some of those comments and discussions.
Nicola: On a few technical forums and one of these post sparked quite a heated debate. It's one of those debates that have occurring themes. It was very interesting to chime in, chat and discuss with other teams and other developers how they're dealing with this sort of thing. A few of the comments that I saw and what is interesting to share with anyone that were recurring. Then maybe we can add our own opinion too and then also say what we think about this. For example, one comment I saw quite a few times throughout this conversations, for example, was, "This is all very nice. In fact, we don't really notice this problem very much because we recommend to our developers always rebase their feature branches before they open a pull request. Then, this problem generally doesn't happen."
This make sense. I think it's a very fair assessment if your team is relatively small or maybe your code base doesn't work very fast. It's absolutely true. If you're just working and you know what the other guys are working on, there's only a handful of branches that working on it at a time, that works perfectly.
It might even work perfectly as the team grows, when you scale this off to bigger projects where there might be five branches every couple of hours popping up and needs to be merge as the tests are complete. You reach a threshold where you merge even enough to see the real state of your branch whenever you would merge it. It's a matter of scale.
Bitbucket and Stash do, they give you a very complete view of what's going to happen if you merge the work of your team's size. You can work around it and a simpler model. It's just slightly more advanced, more complete solution that you get. That was one of my comments.
The other comment that I saw quite a bit. There is this ongoing debate about how to handle the merge strategies, pull request merge strategies. That is, "Yes, but we don't like these merges. We actually like it to rebase the branches and then squash all the feature branchs until we only have one and we'll merge that one. That keeps the history of the project linear, very elegant and clean."
This debate actually has been ongoing for years. There's pros and cons to different approaches. We personally prefer to have a complete view that preserves the context of a feature branch. We're trying to encourage our developers to do explicit merges most of the times.
I can understand in a way that elegance of having a very clean linear history. Some big teams do that too. I actually wrote an article about this a while ago. It's called "Pull Request Merge Strategies" If you go to the website, you'll be able to find it. I don't know if you guys have any other comments on the debate that was sparked on...
Tim: ...It's fascinating to see that level of sophistication of the way that people merge things. There were three or four different kinds of..."Hey, we used merge. We used to do this. We always rebase before we actually pull request. Then, sometimes, we rebase after we upgraded the pull request before merging." It's really interesting to see the plethora of different merge strategies that people fly out there. I noticed that there's a question that just popped up in the little Q&A section. Which version of Stash supports the better pull request? Actually it's been this since the very first version of pull request with since Stash which, I think, was Stash 2.0 from memory. If you're using pull request from Stash then you already have this diff, you don't need to upgrade.
Grace: There's another question as well, Tim. Basically, should you only use the octopus strategy if you want to do a merge where you know that you won't have any tools?
Tim: That's right. I should have mentioned before. If you
want to give it a try, the little Git merge distinct tool that I posted before,
it's actually available via NPM. If you have got node installed locally in NPM,
you can install it and test it out locally. In terms of whether you should use
it when you've got conflicts or not, really the
git merge-distinct adds a sort
of layer on top of the octopus merge that allows you to find which branches you
want to merge. You can still use an octopus merge strategy locally even if you
expect conflicts and then resolve them locally. It has no problems with that.
It's just if you're running it in a continuous integration environment, that's
where you got to want to avoid merge conflicts.
The way that we do that at Atlassian with our developer blog is ensuring that
only branches with distinct paths have been changed. That's built into
merge-distinct as well. In fact, that's something I actually mentioned before.
That's where it actually gets the name.
If it detects the same path has been modified on multiple branches, then it'll ignore later branches. That means that, basically, the only changes to one particular file on one particular branch will be merged into that bundle. Look, I just posted a link to the blog if you want to check it out and give it a try.
I think that really with
git merge-distinct it's only really useful for
deploying changes to types of content that can't really logically conflict. It
works well for our blog because it's static content. It's, basically, a bunch of
marked down files that get rendered into HTML, and they don't sort of interlink.
We wouldn't use this for something like a Java project where you have different class files calling into each other. Because if you'd change the contract of one of those class files and then one branch and you'd change it again on another one or you change something that depend on it on another branch, whether you merge those things together, then you're going to have compilation errors.
You have to be a little bit careful about when you might use the octopus merge strategy for deployment. Typically, static content is a good way to do that.
Nicola: I see there's another question which I think is very interesting. Maybe we can talk about it. From Sam, it says exactly how to resolve merge conflicts and if there's any problems that we could recommend on how to do it and how does that work. Maybe we can elaborate a little bit of it and then give some pointers. Merge conflict is relatively common operation and common issues when you're working with multiple branches and people modifying code at the same time, same lines at the same time. The only thing is you get the conflict is whenever you update changes, you collect and copy changes from other branches like you update the master or you're trying to improve those changes in your own branches if you're rebasing your own local branch or if you're trying to merge your feature branch with master. Then, there's a conflict.
What Git does, it basically annotates so that the very basic way that git does
is it stops the merge operation or the rebase operation saying, "Hey, there's a
conflict that I'm not able to resolve by myself." Git is very helpful. It will
annotate the files. If you type
git status, it will list the files that have
changes both in the incoming branch that you're trying to merge or in your local
If you open those files, you'll see those relatively unfamiliar signs that says, "Hey, this is how that specific line looks in your version and that's how it looks in the other branch." One way to sort those merge conflicts is to do it manually, which I think to get familiar with how git works is a good place to start and you shouldn't be scared by those esoteric signs.
All those means when you see >> it means OK. That is the version that's incoming. Then, you'll see those specific lines look for the incoming changes. The << annotative set of lines are how those specific lines look in your local branch.
What you could do is you can just remove those place holders and make the code. Those lines look exactly as you think they should look. If you need to discuss how things would be resolved, you can always contact a colleague. In general, for say, 90 percent of the conflicts, it's generally easy to understand how things should be merged.
Then, the next step which might be confusing for people is you need to tell git
that you have resolved the conflict on that file. You do that by adding that
file to the index which is to type
git add and the file name on which you will
have just resolved the conflict. By typing git add and the file name that was
conflicting you're notifying git that you have resolved that specific conflict.
You do that operation for all the files that were conflicting. After that, you
can just go on and type
git commit. The merge operation or the rebase
operation that was ongoing will resume. In case of rebase, you have to type
rebase continue so that it resumes. In the case of the merge, you can just type
git commit and then the merge will happen.
That's the most manual way to do it. We have an internal discussion about it just at our company. A while ago, people were asking, "How do you guys solve conflict?" The problem with Sam is a problem that a lot of us have. What's the most efficient way to solve conflict?
We collected a few tips and actually we plan to write about it fairly soon on our blogs so stay tuned for that. Very briefly, a few things that came up and might be useful is you can use...If you use IntelliJ IDEA, for example, and your platform is Java, that has a very solid merging capability that works.
Some people use an open source tool called Meld. Some people use to just to look at the diffs using SourceTree, which is our native client for Mac and for Windows. Some people do it manually. One thing you can do is go online, look for tutorial on solving Git merges and you'll get some of the flows or try to use a visual client.
As you see, Tim has posted a link to our beloved SourceTree. I don't know if any of you guys...Steve, you have anything to add. I just wanted to give a brief overview about this because a lot of people have troubles with solving conflicts.
Grace: Super! Thanks, Nicola. Ian, I'm going to switch over to you in Munich here for a minute. You recently had a nice lesson around git and the new blogging processes. You want to share some of that with us?
Ian: In a way, it's like this reverse process of the octopus merge. The situation that I had is I started with a branch where I created a lot of related blogs just to kind of get a feel for what might be a series. I try to cut them down into smaller post. I was very disciplined about commits, so I made small changes to one file at a time. By the end, I just wanted to pluck out the series of commits that related to the one that is the most mature, ready to go out the door. How should I do that? In the Getting Git Right tour, we talk about rebase as we're playing commits on another branch or cherry-pick. Tim and I kind of bounced a couple of ideas around. I tried interactive rebase. One of the drawbacks there is that interactive rebasing didn't just replay the commits that I was picking onto the target branch. It was actually recreating the branch that I was working on with all the big commits and all of my content. I had to back that out.
I use the main blog and rearrange the pointer, something else that we pointed out in the Getting Git Right tour. This is a good reinforcement. Eventually just went back to cherry-picking. I just took kind of a blog post that was out there about how git rebase is defined in terms of git cherry-picking. Plus, some of the additional things that you would have to do with manipulating an intermediate branch just to kind of get the same effect as the rebase.
Then, that helped me understand where I was kind of losing the plot about rebase. Turn out all the work in the end with cherry-pick and I was particularly pleased with how easily I was able to do the cherry-picking. That's about it right there.
Grace: Super. Thanks, Ian. We've got a few more questions that have come in. Let's see here. What is the best form of deployment from staging to production? Do you have staging as you first stream and then have production as an upstream? I found to be running to this issue lately. Anyone want to take that one?
Tim: In fact the way that we do it in the blog and some other teams at Atlassian where particularly for SaaS projects where we have one staging server and then our production server beyond that is that staging is always a subset of the commits that are on production. We usually do is name our staging branch develop and name our production branch master very similar to git flow. Basically, we promote changes through staging to master. The only thing about this particular way of working is that, basically, any changes that you have on develop should be intended to go through the master.
We try to make sure that our features are complete before we merge them in. It's very rare that we sort of unmerge a branch from develop. Instead, we typically try to roll forward, so if we kind of merge a feature into develop and then there's issues with it, we'll create another feature branch and get that fixed on staging before we promote to master and production.
The actual mechanism that we use for that is typically where we can continuously deploying from both the develop branch and the master branch. Basically, as soon as you merge something to develop, that will get deployed to your staging server. Then, as soon as you merge develop into master, that will get deployed through your production server automatically using Bamboo. I don't know if you guys have anything to add to that.
Steve: My background on my previous team was on the internal systems team, business platforms team. We do a lot of continuous deployment. One thing we definitely try to do was basically roll out onto staging. Then, once we went through certain human-based QA on staging, whatever was on staging got promoted up to master, so there was no real separate streams of development for staging and for production. What we would do, we'd basically...When we're ready to go out and what we'll finally move towards is basically every single branch merged, every single branch, every single feature branch result in a release. It's called release by feature. It gives you very fine grained releases but you have to be able to do continuous deployment at high availability to be able to pull this off.
Every single thing you roll out is single cohesive change not a bunch of unrelated changes. What we do is roll with those out to staging. Once we're happy with them on staging, they will get promoted directly up to master. There'll be no separate staging to master branch. When I say master, it starts in production.
We basically run the binaries for a complete set of automated tests. Roll them straight out onto staging. Staging would go through human testing. Then, whatever is on staging if it was satisfactory would get moved up to production.
Tim: Nice. I just saw there's a follow up question from Ben who asked the original question saying, "We commonly use cherry-picking to redo changes to a new branch when the original branch was made up by an incorrect branch." Look. that's totally fine. Another way to do that is to rebase your branch...Well, I mean, if you branch off master and you want to branch off develop instead, you could rebase your changes with that which effectively a cherry-pick. The other thing you could do is cherry-pick a range of commits. I'm not sure if you're cherry-picking individually or a range of branching. Cherry-pick actually supports the same range of commits back as the get Git-log or Git-diff commands, so you could actually say git cherry-pick the base of the branch dot dot the tip of the branch. Then, it will actually cherry-pick all those commits over to a new branch. Hope that helps. [chuckles]
Nicola: Yes. I saw that. I was about to answer exactly the same question. What I was thinking is that there's no danger in copying commits around as long as you are the source is not used. In reality, if you reapply the exactly the same code changed twice, what you'll get is a NOOP. That's relatively low risk. What is wrong about using cherry-pick is, because you're randomly reapplying patches here and there, you'll lose track of where those changes come from. That's the reason why we try to isolate and then keep work in feature branches, so that it's easy to track where do changes come from. There are specific scenarios like the ones you mentioned that aren't simply acceptable.
Grace: Nicola, actually while we have you on here, do you want to talk a little bit about your Docker Hub hack?
Nicola: Sure. It's one fun hack I worked on a while ago. Every few months we do a hackathon here. We call it ShipIt Days where the company stops for 24 hours and groups are formed and we work on stuff that's cool and could help us solve our own problems and improve timing and things at Atlassian. My idea was, "Hey, I'm very excited about Docker stuff. Can I do something relatively quick in 24 hours that can have an interesting impact or interesting use using Docker?" I wanted to do something with Docker using Docker. [chuckles] What I did, maybe I can share the pages I talked. There's an article about this on the blog. It's called Hacking the Docker Hub into Bitbucket. Docker has a registry where you can prepackage your applications to run in containers, it's registry.hub.docker.com. There you can see the major data and the amounts of downloads of a certain container, certain package application has.
I thought you can build containers from Bitbucket, but wouldn't it be nice if it can show actually that information in Bitbucket itself? How many people have downloaded that container if it is packageable as a Docker image?
What I did is I asked friends in the Bitcuket team to open an iframe there and I developed a small static website based on nginx that would go to the Docker Hub and collect all those statistics and inject it inside Bitcuket. That's rolled out into production.
I managed to do that all in 24 hours. It's only limited as for exactly that. Actually, you cannot go to Bitbucket and see it live. I put a video of it demoing it and showing that it's actually real and you can do a live update of this platform just by using and starting containers. I encourage you to have a look at the video and I posted a link.
I'll post the link into the chat if you guys are interested. I thought it was an interesting hack to show. Because we're very interested at Atlassian about new developments about Docker, new trends of Cloud-based deployments. Personally, I'm very excited about it.
Grace: Awesome. Thank you, Nicola, for sharing that with us. That wraps up our content for this session of the Dev Den Hangout. Thank you everyone for joining us. This is being recorded so we'll posting this recording for everyone so you can take a look at it again. We'll have a transcript available for it in two weeks. Thanks everyone. Thanks guys.
Nicola: Thank you.
Tim: Bye guys.