[00:00.000 --> 00:03.920]  Hello and welcome to this new edition of the FUJAY Podcast.
[00:03.920 --> 00:06.840]  Today, we are wondering what the future of source control
[00:06.840 --> 00:09.160]  and build automation will bring us.
[00:09.160 --> 00:12.240]  Welcome to the FUJAY Podcast.
[00:12.240 --> 00:15.360]  All your news about OpenJDK.
[00:15.360 --> 00:17.040]  As developers, we like to automate
[00:17.040 --> 00:19.120]  the boring parts of our job.
[00:19.120 --> 00:21.560]  The source control system, build pipelines,
[00:21.560 --> 00:23.240]  and code analysis tools we use
[00:23.240 --> 00:25.920]  are crucial parts of this automation.
[00:25.920 --> 00:28.320]  And like almost everything in the software world,
[00:28.360 --> 00:30.680]  these parts are constantly evolving.
[00:30.680 --> 00:33.480]  We've switched from CVS to Subversion
[00:33.480 --> 00:35.760]  and from Subversion to Git.
[00:35.760 --> 00:38.520]  And we started to embrace trunk-based development
[00:38.520 --> 00:40.080]  instead of creating branches
[00:40.080 --> 00:42.080]  for every feature that we work on.
[00:42.080 --> 00:44.240]  So is this the end point?
[00:44.240 --> 00:48.160]  Did we find the holy grail in version control already?
[00:48.160 --> 00:49.920]  And what additional evolutions
[00:49.920 --> 00:52.040]  are perhaps still waiting for us?
[00:52.040 --> 00:56.360]  Let's find out together with these amazing guests.
[00:56.360 --> 00:57.640]  Hello, I'm Tricia.
[00:57.640 --> 01:00.360]  I am a lead, what's my title now?
[01:00.360 --> 01:03.240]  Lead Developer Evangelist at Gradle.
[01:03.240 --> 01:05.600]  I joined Gradle in January.
[01:05.600 --> 01:08.000]  Before that, I was working at JetBrains
[01:08.000 --> 01:09.880]  doing Intelligent Idea Advocacy.
[01:09.880 --> 01:13.160]  And last year I had a tiny break to write two books
[01:13.160 --> 01:15.880]  because that's what you do when you want to take a break.
[01:15.880 --> 01:18.560]  So the thing, the kind of thread that kind of follows
[01:18.560 --> 01:20.600]  all of my work in, especially in developer advocacy
[01:20.600 --> 01:22.560]  has been sort of developer productivity.
[01:22.560 --> 01:25.480]  How can we use our tools to be more effective as developers?
[01:25.480 --> 01:26.840]  How can we automate the way
[01:26.880 --> 01:28.480]  the kind of boring stuff,
[01:28.480 --> 01:31.440]  which is why it's really interesting to talk about
[01:31.440 --> 01:34.960]  version control and CI, CD and things like that.
[01:34.960 --> 01:36.440]  Hi, I'm Ixchel Rees.
[01:36.440 --> 01:38.480]  I'm a Developer Advocate at JFrog.
[01:38.480 --> 01:42.720]  This is my first time being a Developer Advocate.
[01:42.720 --> 01:46.640]  I have been a consultant for the last 20 years.
[01:46.640 --> 01:50.120]  So before I was all about developing software
[01:50.120 --> 01:54.320]  and now it's about sharing, learning
[01:54.320 --> 01:59.320]  and finding new ways of bringing different paradigms
[02:00.120 --> 02:02.840]  into developers and also bringing back
[02:02.840 --> 02:06.600]  the feedback from developers to the different companies
[02:06.600 --> 02:10.240]  because that's also really, really important for us.
[02:10.240 --> 02:13.520]  And I'm super excited to be here.
[02:13.520 --> 02:16.600]  So thank you very much everybody for having me.
[02:16.600 --> 02:19.320]  Hello, my name is Pierre-Etienne Monnier
[02:19.320 --> 02:22.720]  and I guess I was invited here
[00:00.000 --> 00:03.680]  into developers, and also bringing back the feedback
[00:03.680 --> 00:06.520]  from developers to the different companies,
[00:06.520 --> 00:10.160]  because that's also really, really important for us.
[00:10.160 --> 00:13.520]  And I'm super excited to be here.
[00:13.520 --> 00:16.480]  So thank you very much, everybody, for having me.
[00:16.480 --> 00:17.680]  Hello.
[00:17.680 --> 00:19.200]  My name is Pierre-Etienne Monnier,
[00:19.200 --> 00:23.000]  and I guess I was invited here because I'm
[00:23.000 --> 00:28.000]  the author of one of the main authors of Bifu, which
[00:28.000 --> 00:31.360]  is a version control system, sort of a hipster version
[00:31.360 --> 00:35.400]  control system based on new ideas from mathematics.
[00:35.400 --> 00:41.480]  It starts as an improvement on darks, which was itself also
[00:41.480 --> 00:45.840]  based on some fuzzy theory of patches,
[00:45.840 --> 00:51.960]  and we've expanded it to make it super fast and easy to use,
[00:51.960 --> 00:53.480]  scalable.
[00:53.480 --> 00:56.800]  So before that, I'm working on this now,
[00:56.800 --> 00:59.720]  and before that, I was an academic researcher
[00:59.720 --> 01:02.280]  on a variety of topics, mostly related
[01:02.280 --> 01:04.840]  to AC-produced computing models.
[01:04.840 --> 01:09.280]  Before COVID-19, I was working on molecular computing,
[01:09.280 --> 01:12.200]  so how to build molecules out of DNA,
[01:12.200 --> 01:15.120]  how to program matter to assemble itself.
[01:15.120 --> 01:21.080]  And since COVID, now I'm working mostly in the energy sector,
[01:21.080 --> 01:23.520]  trying to help people share renewable energy
[01:23.520 --> 01:25.280]  with their neighbors.
[01:25.280 --> 01:29.320]  This is also an interesting theoretical problem,
[01:29.320 --> 01:31.120]  because you have a shared resource,
[01:31.120 --> 01:35.880]  and you want people to get enough to feel
[01:35.880 --> 01:38.080]  that they got enough, and at the same time,
[01:38.080 --> 01:41.600]  to optimize profitability of the setup.
[01:41.600 --> 01:44.760]  And I'm also working on Bifu with that,
[01:44.760 --> 01:47.160]  so lots of things in parallel.
[01:47.160 --> 01:49.440]  Thank you, and I am Hanno Embrecht.
[01:49.440 --> 01:51.480]  I'll be your podcast host for today.
[01:51.480 --> 01:55.560]  I am an IT consultant at InfoSupport in the Netherlands.
[01:55.560 --> 01:57.840]  And when you're in consulting like me,
[01:57.840 --> 01:59.760]  you tend to get a lot of opportunities
[01:59.760 --> 02:02.160]  to work on different build automation setups
[02:02.160 --> 02:03.800]  and virtual control systems, which
[02:03.800 --> 02:07.240]  is one of the reasons why I developed a special interest
[02:07.240 --> 02:08.640]  in both topics.
[02:08.640 --> 02:11.360]  So thank you for being here today.
[02:11.360 --> 02:15.720]  Lots of interesting angles I think we can take.
[02:15.720 --> 02:18.240]  I think the one thing we can agree on here
[02:18.240 --> 02:21.840]  is that as developers, we like to spend most of our time
[02:21.840 --> 02:24.240]  on actually writing code and delivering values
[02:24.240 --> 02:25.280]  to our customers, right?
[02:25.280 --> 02:29.960]  So that means the more we automate our CI-CD process,
[02:29.960 --> 02:31.360]  the better it is, right?
[02:31.360 --> 02:36.960]  Or are there also any downsides in automating everything we do?
[02:36.960 --> 02:40.040]  HL, I wanted to ask this question to you.
[02:40.040 --> 02:42.800]  What do you think about automating as much as we
[02:42.800 --> 02:44.760]  can of our CI-CD process?
[02:44.760 --> 02:48.200]  Well, that works perfectly fine if we
[02:48.200 --> 02:50.560]  are thinking about the happy part.
[02:50.560 --> 02:57.000]  Because then it's a matter of doing the repeatable things
[02:57.000 --> 03:00.040]  that we know the inputs, we know the outputs,
[03:00.040 --> 03:01.680]  and we know the process.
[03:01.680 --> 03:05.440]  So actually, by adding the human factor there,
[03:05.440 --> 03:08.600]  we probably are adding more uncertainty.
[03:08.600 --> 03:12.600]  And we will be actually the part that
[03:12.600 --> 03:14.640]  is going to bring the errors.
and we know the process.
So actually by adding the human factor there,
we probably are adding more uncertainty
and we will be actually the part
that is going to bring the errors.
There are other tasks by the nature
that they cannot be so easily
to either establish or automate.
For example, if we don't have enough knowledge
or it's so rare,
then the effort of trying to automatize
that it's higher than the return of our investment.
And then are the tasks that are done
to create new knowledge.
Like we don't even know what the input is or the output.
We're trying something completely new.
So as you see,
when we're talking about one side of the spectrum,
the ones that input outputs are known
and the process is well-developed,
we know what we want,
then for sure we should, it's our duty
because that is going to remove
a lot of the uncertainty or problems
that we have with technology.
And most of their tasks are like that.
But no, is everything feasible to be automated?
No.
Should we?
No, I think, I hope I kind of answered that question.
Yeah, sure.
Yeah, sure you did.
Because I fully agree with you.
So Tricia, you told us that one of the fields
that you're interested in is developer productivity
and HL already mentioned the human factor, right?
Well, developers are also humans, obviously.
So could you tell us maybe something about that?
How do you see the human factor working its way
into developer productivity,
or at least maybe having a detrimental effect to that?
At Gradle, we're talking a lot
about developer productivity engineering,
which is about automating all the stuff
that can be automated, basically removing toil,
some of the stuff that Sher was talking about
in terms of the stuff that's repeatable,
the stuff that's predictable,
that lends itself well to automation.
And so we're talking about DPE being the automation
of the stuff that adds friction, increases toil,
which means things like, obviously, automated tests,
automated deployments where possible,
being able to do things like,
the tool that we have allows us to speed up our build
with both Gradle and Maven
so that the build doesn't take so long,
running the tests don't take so long,
all these things that you end up waiting for as a developer.
That allows the human to do more stuff
that the human's good at, right?
So if you're not waiting 20 minutes for a build time
for your build and tests to run,
and you are getting fast feedback from your tools,
then you can be thinking about the next step.
And what developers are really good at is being creative.
So of course we type stuff into the computer,
of course we tell the computer what to do,
but the difficult bit is not that bit.
The difficult bit is, how do I solve this business problem?
Have I asked the right questions?
Have I thought about all the angles?
Have I thought about the way the user
is definitely gonna add the wrong thing
and it's going to break?
And machines can't do that.
So it's a long way in terms of being able to do,
generate various tests, which can throw things at,
throw rubbish at your application.
You've got like, what's it called?
Like chaos monkey stuff and fuzz testing
and all those things.
You still need a human to kind of make the decisions
about what should the application do?
What does it look like when it goes wrong?
When is it likely to go wrong?
And so for me, developer productivity engineering,
and which is aimed at improving developer productivity,
is about automating stuff that should be automated.
I talk about this when I talk about code reviews as well.
Like code reviews, you should not be looking for formatting.
You should not be looking for anything
which a linter can do or anything
which an automated test can do.
A human is not good at that.
So if you automate stuff that is automatable,
then the human element comes in in terms of,
are we building the right thing?
Are we building it correctly?
Have we tested the right things?
Have we covered all the areas that, you know,
we wouldn't have thought of?
Instead of spending, and I've been in jobs like this,
instead of spending three hours running a manual process
to release to the test environment every day,
you get to spend those extra three hours
writing good automated tests,
or having conversations with the business
about the problems you're trying to solve.
And the thing that a developer should be really good at
is being able to take these requirements,
start working on things, but then go, oh, what if?
And then being able to take those questions
back to the business.
Because those are the sorts of things
that maybe the users in the business haven't thought about.
Oh yeah, it's true that we haven't really thought
about this angle, or you as a developer are thinking,
well, we have a module which does this,
and you want a module which does this.
What if the interactions worked like this?
How would that impact the business or the user?
And that's what we want to free the developer up to do.
We want them to stop thinking,
stop spending time on stuff which is like high friction,
like low value, and allow them to start
bringing the good human stuff to the process.
Yeah, we tend to produce still some good human stuff, right?
There is the human factor,
but there can also be benefits to that.
Pierre Chen, do you recognize some of these challenges
that a lot of time can get wasted
waiting for builds or test results,
and how do you typically tackle them?
Yes, yes, absolutely.
So it's interesting to hear
the industrial developer perspective,
because I'm not an industrial developer myself.
I don't work on huge projects with very large teams
and salespeople come at you and say,
well, the customer wants that, and this, and so on.
So yeah, I definitely recognize some of the challenges.
So one thing that came to my mind
as I was listening to Ikshela and Trisha was
that sure, automation is important,
and good for you, and good for everybody,
but not just any automation.
For example, one common pattern is that
you have a problem that's completely artificial
that could be solved by just
not having the problem in the first place,
and you're like, okay, let's automate that.
Let's automate the solution.
For example, in developer development workflows,
sometimes you're like, well,
when you're using the wrong tools,
like for example, in the vacuum today
where everybody was using SBN or CBS,
some of the problems was locks on files.
And so people were wondering
how to automate the way to take a lock on the file.
And then we all have asynchronous distributed
virtual control systems,
and we're like, oh, this problem no longer exists.
So many times just being clever goes a long way
before we actually need automation.
So that's one different perspective
I want to bring to the table.
Many times, yeah, automation is good, sure,
but sometimes just thinking deeply about the problem.
Modeling it really carefully and making sure that,
and challenging it is important as well.
For example, sometimes you have a model
and that's the first thing that comes to your mind,
and you're like, oh yeah, this model is really good.
It took me a lot of effort to produce it.
And you face challenges, and you're like,
oh yeah, let's go with that.
And then you encounter lots of problems along the way.
That certainly you can tackle with automation,
but also by thinking a bit more
or challenging your models a bit more
before starting out,
you would probably not have even met problems.
And another comment I had about something both of you said
is that one of my use cases
was French National Assembly.
So the thing that made me realize that was the comment
on the human factor and what should be and should not be
automated.
So the French National Assembly is like any parliament
in the world, is a typical example of version control.
Bills that come to parliament are very rarely new laws.
I work completely out of the computer programmer realm
here, right?
So bills are very rarely new laws.
Usually they are written as a patch to existing law.
And they're really written in that form.
If you look at bills, you're like, oh, change,
replace words, replace every occurrence of words,
through by words bar.
And that's like all bills in all parliaments in the world,
like most parliaments have looked at at least,
are exactly formulated in the same,
with the same, almost the same language,
or at least a natural language version
of what we do with Unix diff.
And so you have these highly qualified lawyers,
and here comes the human factor, right?
You have these highly qualified lawyers
who take the amendments to the bill,
so change patches to patches.
So they take these amendments every week,
and then they try to refine the conflicts
between the amendments.
Then they present them to votes, right?
So voting is like a trend-based development.
And then they manually apply them to,
otherwise they resolve the conflicts
by not presenting to votes amendments
that would conflict with the currently voted amendments.
So while they're in a session in parliaments,
they have to keep track of,
manually keep track of what amendments were applied,
and how they were applied,
and what that changes for the next amendments, and so on.
That can completely be automated,
not up to the last sentences
where people have to actually know
what they've written makes sense, legally speaking, right?
But a large part of it could be automated.
And then after the vote,
these people look at their law written in a word document,
and then they manually apply the amendments to the bill,
and then the bill to the law.
And so you have these highly qualified,
highly paid lawyers paid to do,
well, certainly interesting, relevant legal work,
but also manual version control.
There's no CICD as well in parliaments,
and we could have that.
Imagine, for example, for the taxes,
we have a budget every year.
This is a perfect test case for taxes.
How much are we going to guess in tax money?
How much are we going to spend?
If I change the percentages on something in some law,
what impact will it have?
So all that kind of stuff,
this could also be completely automated,
and we're not doing that.
But anyway, so that's quite far from developments.
It was just to say, well, sometimes the frontier
between what can be automated and what cannot be
is strange and interesting,
and that's always an area of wonder, I think, in our job.
It's like, how far can we go in automation?
And can we solve the problem by just modeling it better,
or do we actually need to run big machines
on huge test sets?
I think it goes without saying
that a lot of stuff can be automated in general in the world.
Just the fact that we are software engineers
and we tend to have some experience with automation
means that we are right there at the right place
to automate it.
But of course, if one of us has the ambition
to transition into French law,
we can maybe apply the automations there,
help the French parliament into the future.
We're not using our parliament anyway these days,
because the executive passes the bills
without going through a vote.
So that's a perfect test case for people like us, right?
Right, you mentioned some interesting things, Pierre Chen,
and it made me think about
using distributed version control right now.
And I think most of us are using Git,
but you can elaborate on that
when I ask you the question later.
The human factor is also a factor
when you use a virtual control system, right?
I have seen a lot of coworkers struggling
with their Git repositories.
And I was wondering, Ixchel,
maybe what virtual control system do you actually prefer?
And what have you seen out there in the wild
when your coworkers used version control,
that they also kind of wasted some of their time
because they ran into some problems there?
In terms of tools, what I have seen out there,
actually I started with subversion.
At the beginning of my career,
I actually had to deal with it.
I have a very rare encounter with Mercurial
in some of my customers,
but most of them, and since some years now,
most of the people that I encounter, developers,
or companies that I collaborate with,
it was mostly Git.
Git is an improvement into what we had before.
I mean, having complete code base repeated,
different centralized ways of sharing the source code
was a little bit complicated.
And then for example, having to resolve all the conflicts
before making a commit, instead of Git,
committing your actual work
and then trying to figure out the conflicts.
It's also an improvement.
How do you see your work
and how do you see yourself collaborating with source code,
a large source code?
The challenges are obvious.
I mean, at least for me,
our code base are increasing in size
in such high speed pace
that we need actually tools that are easy to use
and can keep up with all this code bases,
extremely large code bases,
and with different kind of binaries or files.
Git, it is very efficient for some things,
but also allows people to get really frustrated
as you mentioned, Hanno.
I have seen, and I have people refer to Git
and how do you get from point A to point B
and saying, well, you can get from point A to point B
like a snowflake, totally different ways to do it.
And I have seen people that use the reflog,
like their best friends.
I see people that don't even understand
what the reflog is.
I see people going and driven crazy
because of all the merge conflicts.
And when they are seeing the merge conflict
and says, okay, three commits of 180 to come.
And they are like, oh my God, what should I do?
I have seen bad practices, rewriting history.
I have seen so complicated workflows
that you are like, what were we doing in the first place?
Like, I mean, we've been working on trying to fix
the state of the files, but at this point,
I even forgot what we're trying to achieve here.
Right now, I think Git is the most popular one.
It's very useful, it's very powerful.
I think it's the industry almost default.
I think you still can misuse the tool,
and I think there is still a lot of unknowns.
There are still a lot of best practices out there,
but I still think that the level of usage of Git
in the normal developer is not proficient.
So I still think that we have UI interfaces
that obscure a lot of the main knowledge of Git
and as a developer in my perspective
and in my way of behaving, I like the common line.
So that gives me a lot more control.
I know, Trisha, I know, I know.
But at the beginning, it was the best way
to actually tame down that beast.
Anyway, I hope I answered your question, Han.
Yeah, you did.
Actually, this really rings a few familiar bells for me.
So at Info Support, I teach a few courses,
and one of them is Git for Developers.
It's one of the first courses that are the people
that start out at our company do.
And I always try to teach them to use the command line.
But also, one of the first things I show them
is a wonderful website that is called dangitgit.com.
We can add it to the show notes.
It's a very funny website created by Katie Seiler Miller.
Dangitgit.com, and it's just a knowledge base
of stuff you can do to fix up your repository
when you did something wrong,
because doing something wrong with Git is quite easy,
but then fixing it, well, that is just a bit harder.
And it's just a knowledge base for Katie
to be able to refer to it back again
when she encounters a similar issue.
So I completely agree and recognize the things she said.
Trisha, what are your experiences here?
I mean, do you like the Git command line?
I got the feeling from Iqchel that you don't really,
but you can elaborate.
I mean, Git is my default as well.
I mean, everyone uses Git these days.
I actually agree with a lot of what Iqchel says.
I don't mean this as a criticism of normal developers
or of like the majority of developers.
When I say many developers don't know how to use Git properly,
it's not their fault.
It's that the tool is so powerful, so huge,
has so many different variations,
is used differently by almost every team
in the whole wide world.
It is impossible to know the whole tool.
It's impossible to know how to use it properly.
One of the things, a slight plug for my own book.
One of the things I tried to teach
in the IntelliJ IDEA book I wrote last year,
this comes back down to the command line versus the UI.
So IntelliJ IDEA has a really nice UI
for working with version control.
I'm not really a command line sort of person.
So I do prefer using the UI.
When I was trying to indirectly teach version control
while teaching it through the medium of IntelliJ IDEA,
the idea is just to try and expose developers
to the smallest amount of stuff
they will almost definitely need to use.
For example, committing, pushing, branching,
merging, what's a merge versus a rebase.
I don't know if I added cherry picking.
I think I might've decided that, let's not cover that.
There's a lot of information about Git out there.
There's a few great new books out there as well,
including Headfirst Git and another book,
whose name I can't remember,
but I'll try and give it to you
so it can be in the show notes.
A lot of great new resources on Git.
But for the longest time,
a lot of people were trying to teach Git
from the command line in terms of teaching,
here are all the features, pick and choose the ones you want.
Cherry pick the things that you want.
And I think what we really needed to have taught developers
is here is your entry level Git.
Here is the level up.
And then from there, this team uses Git in this way,
and these are the things that you will use,
and this is how you will use it.
And I think that way we can get a bit less confused
over what's a rebase versus a merge
and when do I use these various features.
I think that it just becomes very complicated
to understand these things.
However, I'm also a big fan of chunk-based development,
and I'm not a big fan of long-lived branches.
I don't like the idea of having a long-lived feature branch
which needs merging or rebase,
and rebasing a long-lived feature branch
is like the most difficult thing that you can do.
You need to do regular merging between various branches.
I don't like the idea of having a branch of work
which is you might be running your tests on it,
but you're not doing continuous integration
because continuous integration means integrating it,
not just running your tests on it.
So these are some of the other patterns
that you can have in Git that I personally don't like
because I think it pushes a lot of the pain
to the end of the development process
by which you get to the point like it was saying.
By the time you start merging or rebasing
or trying to actually integrate your work,
you've forgotten what you were working on.
The code base has moved on from where you started.
You're not really sure how your changes
are impacted by someone else's changes.
And I have often seen people going,
I don't know what this diff means.
I'm just gonna take my changes
and trample all over someone else's,
or I'll take all their changes
and just rewrite the whole thing.
And this is when we get back to manual processes
and toil and pain.
I think that some of these tools can make it too easy
to do things which make developers' lives difficult.
But again, it's all about the people.
It's all about the team.
It's how the team chooses to work.
And it's about having a consistent process
across all the people in that one team
and understanding how things are done
and also regularly integrating in my personal opinion.
Very familiar things that you brought up there.
So Pierre Chen, I was wondering, of course,
I know you from the Puchul virtual control system.
And I was wondering if the things that Trisha just told us
also inspired you a bit to start working on that.
Long-lived feature branches and conflicts
that are very hard to merge.
Literally every single sentence said by HL and Trisha
in the last five minutes were big triggers for me.
I've had to take notes, actually.
So there's a lot for me to comment on.
I'll try to be brief, but it will be hard.
Well, I actually said something
that I found really very relevant.
She said that few developers are proficient in Git.
So how is it even possible that a tool that's 18 years old,
that's widely used by everybody in the world,
is in the state where a few people are proficient in it?
I'll actually elaborate on that and go a bit further.
How is it possible that nobody outside our industry
uses Git?
It's supposed to be one of the great prides
of computer developers that we've been able
to build a tool that is this powerful and useful
and relevant to all kinds of projects,
while accepts none in the actual world.
So it's relevant to all projects,
but it has to follow good practices.
So good and bad practices were repeated many times
by both HL and Trisha.
So yeah, good practices.
So what's a good practice?
Let's say I want to build something with my Lego set.
For example, I love playing Lego, right?
So I want to build, let's say, a car in Lego.
So what's a good practice for that?
There's no real good practice.
Like, we should just play and let the tools do
what they're meant to do.
Like, if I want to use a screwdriver on a screw,
so what's a good practice for that?
Well, just use it.
So it's striking to me that we've not been able
to export our technologies to outside
on a little tiny field.
Like, well, computer science is not exactly a tiny field,
but anyway.
For example, these days, who doesn't use stuff
that was invented at NASA, for example?
Everybody uses NASA-invented stuff,
like from silverware to outdoor equipment,
to like anything, really.
NASA is really proud of it and communicates heavily with us.
Who uses stuff not in computing
that is invented at, yes, nobody.
I think that's striking.
And it's not because the tool is too young.
Tool is like 18 years old, 500 different commands,
like subcommands.
And that's something I think,
I remember I think Trisha said,
it's like there's lots and lots of commands
and nobody really uses it,
like knows every single command.
One link I've posted in this chat
is the gitman page generator.
It's a website that generates a new man page
for gits randomly every time you click on it.
The man pages are obviously completely fake,
but they sound real.
Like when you read them, you're like,
oh yeah, this could definitely be a git command.
This is incredibly complicated.
So yeah, what inspired us to start
was that we needed a tool that would be,
that would allow us to do everything git can do,
but also be simple.
And while I mean simple to use really fast,
that was a requirement.
And that was let people just work
instead of teaching them how to,
like teaching them the good practices,
telling them whether to merge, to rebase, to cherry pick.
Well, all these three commands in people are the same one.
You just apply a patch.
And that's actually what you want to do.
When you're rebasing stuff,
you're just using patch commutativity
to swap the order, for example.
So, and that's the default in people here that you're,
it's the default because you don't need to swap the order.
The order is irrelevant.
If you can't apply the patches in two different orders,
you don't need to swap the order because it's the same.
And that thing, even if you don't swap,
if people, if you don't use,
if you don't think you're using patch commutativity,
you're still using it, you're still benefiting from it.
So just some context here.
So people is a tool that's based entirely on patches
instead of snapshots.
So commits are snapshots.
And Git rarely uses patches.
And in people, we do the opposite.
We focus primarily on patches
and the user only sees patches.
And then behind the, like under the group,
we're using a giant data structure,
a graph-like data structure
that's used to merge all the patches.
So the patches are not exactly Unix patches.
They have a little bit more information,
but we're using a data structure that's actually a CRDT.
It wasn't found in this way.
It was found by asking the question.
It was a purely theoretical question.
We want to handle changes on files, right?
So what is this?
So, okay, we can have conflicts sometimes
and conflicts are bad and nobody likes them, right?
So what is the question?
The initial question was,
what is the minimal extension to files
that can tolerate conflicts?
What does it mean to tolerate conflicts?
So asking this question means,
what is like, you can merge your changes.
You can apply your patches all the time, always.
Sometimes you will have conflicts.
And because you've asked that question mathematically,
you know that your data structure
will not have conflicts, probably.
So you can apply your patches.
You will have conflicts.
And at the end of the sequence of patches you've applied,
you can ask the question, okay, where are my conflicts?
And then you can solve them.
You solve them by creating, guess what, a patch.
And when you send that patch out to others,
we may have the same conflicts.
Well, it solves them for them, too.
So you don't have to worry
about whether my branch is long-lived, short-lived,
or a feature branch.
We are just using branches and patches.
And that's, it's much simpler.
You have way fewer concepts.
And you can do the exact same things
while, like I said, producing the bad merges in Git,
which you can very easy to get,
even by following good practices.
Yes, all these issues, the two reviews I've mentioned,
were definitely a starting point for us.
Yeah, thanks, Pierre Chen.
So quick follow-up question here.
I'm assuming you use PyHool yourself.
Yeah, I don't use any other version control system.
Yeah, yeah, yeah, I'm not surprised.
You said you can either choose to use short-lived branches
or long-lived branches with PyHool
that it would be just as easy.
So what is your preference there?
Well, it depends.
I don't use branches too often, to be honest.
My favorite model of branching is patches.
The reason we use short-lived branches or feature branches
or trunk-based developments
is that we want our different features
to be independent from each other.
And why do we want to do that?
Because we want to be able to maybe push them
independently from each other or maybe undo them, right?
And when you have a system that handles patches properly,
this is something you can do even without the issues.
So for example, in people,
let's say you have two different patches
that introduce features,
and you would call them branches,
and it would require you to plan your work ahead of time
to say, well, now I'm working on this feature.
But that's not all the way I work.
Well, maybe it's not suited for everyone,
because that's not all the way I work.
Usually when I start working feature,
all things are broken always,
and I discover a bug,
which leads me to believe that I should be working
on another feature instead.
So in GitHub, you have to use all the arsenal
of stashing and staging and whatnot, right?
But in people, you can just write patches.
Well, this is the only command we have,
so you're bound to use it anyway.
So creating a patch, applying a patch, that's all we do.
And so, yeah, my favorite model of branching is patches.
So you're working on two features in parallel,
just make two batches,
and then you'll be able to push them independently,
and you'll be able to unrecord, undo them independently.
If they're not, if they're separable, actually,
so if they depend on each other,
if they touch the same place in the file,
then you will have to be a bit more rigorous
and just meaning to just do it
and then solve the conflicts afterwards, if you want.
Like to do the actual branches,
we do have branches in people,
but you can also use patches for them.
And so another model of branching,
which I've also used sometimes,
is you want to maintain two different versions in parallel.
Like, let's say, for example,
the feature in the language you're using
is not quite ready,
but you want to still experiment with it,
and all the bug fixes you find while experimenting with that,
you want to back-prompt them to your branch.
So there you have two long-lived branches,
and they're necessary for your users, for your customers,
because your customers want to be able to,
like two different classes of customers
want two different versions.
And then you'll be able to just back-prompt,
you won't have to back-prompt,
just have to take the bug fixes,
take the bug fixing patches
and just apply them to your previous branch.
And that's it.
And then when you merge everything later,
because the patch, when you apply them,
don't change their identity,
you'll still be able to merge all the other changes,
and you won't get any merge conflicts,
because the patches aren't the same.
Like, the patch, one single patch is the same
no matter where it lives.
It can be applied to different branches
in different contexts.
That won't change its identity.
So you will still be able to merge the other,
like the new experimental branch
without losing your mind on conflict resolution
and all that kind of stuff.
Tricia, you also mentioned trunk-based development,
just as Pierre Chen did a few moments ago.
One thing we could do is switch to Pekul.
I'm all for it,
but I still have to convince my coworkers
and the client side that are working.
But I think a few of the trunk-based development ideas
can still be applied,
regardless of what virtual control system you're using,
right?
I wrote a blog post on it a couple of weeks ago.
When I worked with Dave Farley,
who wrote the Continuous Delivery book,
we were doing, well, to be fair,
we were using Subversion,
but even if we hadn't been using Subversion,
we would have been using trunk-based development.
The idea being that everyone is working against trunk.
They don't have their own branches.
They're not doing feature branches.
The only branch you have is that the code
that's on your machine is separate to the main trunk.
And it works in Git as well.
You make some changes.
You make sure everything's compiling.
You make sure all the tests are running.
You commit it to trunk, to the main branch.
And then that way,
because your CI is working all the time on your trunk,
you know that all your acceptance test paths,
all of your performance tests meet the demands.
And your trunk is always ready for deployment as well,
because this is an important thing
for continuous deployment.
The idea being that any point in time,
you take what's on trunk, all the test paths,
and you just deploy it whenever you want to.
When I was working this way with Dave Farley,
it was a few years ago,
and we were doing releases every few weeks.
And I know nowadays,
lots of people are doing microservices
and they're releasing potentially multiple times a day.
But to me, that doesn't mean that you can't still do
a development model where you are pushing your code
into trunk all the time.
If your feature's not ready, it should still be compiling.
Even if it takes you two weeks to create a feature,
because for example, I was working on a financial exchange
and some of these features take quite a long time
to implement new instruments
or new ways of working with the account.
But you can be using feature toggles
or literally just code paths that can't be reached
until you actually connect them up via a button on the UI
or whatever it is.
But all the time that you're working on this feature,
even if it's a long lived feature,
you're still working on the main code base.
You're still not merging the main code base
into your code base.
The only way you're merging it is when you do a get, a fetch,
and you know that your code is always compiling
and you know that your code will continue to compile
against trunk because you're constantly working
against trunk.
You're also getting all of the other developers fixes
and changes as they work against trunk too.
Now, I did write a blog post about this recently.
I understand there are, like with anything,
there are lots of it depends and criticisms
and places where it doesn't work.
So for example, if you've got 300 developers
working on a huge application,
which is all in one module on trunk,
maybe you're gonna end up with a lot of problems
and you're gonna be triggering an awful lot of CI builds
on the same branch, on the trunk.
However, I think if you've got 300 developers
working on one large mono repo,
I think you've got other problems as well.
I think the movement towards microservices came
because now we have a lot of big code.
We have, like I think Michelle mentioned this as well,
these code bases are getting bigger.
You've got more developers working on code.
One of the ways to address that is to split it up
into separate services or separate modules,
separate teams.
So no, of course, something like trunk-based development
doesn't necessarily scale if you've got 300 developers,
but if you've got 300 developers,
you have a scaling problem anyway, probably.
So yeah, so my preference is that
the core of what you're working on,
be it a microservice or a module or a project,
all the developers who are working on that
should be committing to that main trunk.
The only real exception is something
that Pierre-Étienne mentioned,
which is that sometimes you might have the branch
that's in production and the branch that's not in production.
So I think that's the exception there.
So Tricia, what would be your advice for people
who are currently in a feature-based approach,
but really want to transition to trunk-based development?
And also maybe second question,
maybe you can address it as well.
What if the context you're working in on a platform level
demands that you use feature branches
because it's linked to tickets, ticketing system IDs
or something to give some advice on that?
I've been thinking about this a bunch.
And we have to bear in mind that
I've been doing developer advocacy
for the last like 10 years or so.
So maybe I'm a little bit divorced
from the reality of real developers.
Having said that, the flip side is
I do quite a lot of open source development,
which means that I often wander off for weeks at a time,
don't do any code and then come back
to an open source project.
And of course it's moved on since I did some work.
And one of the most important things
in these environments is to continuously pull
from what's new into your branches, into your fork.
And I think this is one of the things
that I have seen go wrong in long-lived feature branches
is that people start a feature branch
and I've worked in teams like this.
You start a feature branch when you start that feature.
And like you said, you've got a tracking number
for the issue number or the bug or whatever.
And you start working on it today
and you finish it next Thursday.
And during those two weeks,
you haven't pulled from main at all.
And that's when, and you haven't merged anything,
any of the other developers stuff into your branch.
And that's when you start getting like merge hell
because you merge it, you may hopefully merge it
at some point before you do your pull request
or your merge or your rebase.
If you're regularly fetching from what's in trunk,
then you end up having smaller,
lots of changes that you need to pull into your code.
You have fewer chances of merge conflicts
and you're working on the most up-to-date code,
which means that as soon as someone has worked on something
which is going to conflict with yours,
you see it sooner rather than later.
So one thing is I would definitely pull more regularly
from a trunk if you have to have long-lived feature branches.
And the other thing is to try
not to have long-lived feature branches.
It's really important.
And this is a computer science problem since forever.
This is not a good thing.
It's important to think about things
much smaller pieces of work than we generally have been, right?
You shouldn't be working in isolation on one thing
for two weeks.
I mean, a lot of people are asking me after the blog post,
what's a long-lived feature branch?
Anything longer than 24 hours, really?
Because you can't be working in isolation
for that amount of time.
You need to be able to split stuff up.
If you're working on something super separate,
it's kind of easier in a way.
You created a new instrument class, let's say,
for the particular instrument that you're creating.
You can commit that and you merge it into main
and nobody's using it.
So it doesn't really matter.
There's no impact of that.
So small changes, very small changes.
And if there's no conflicts and nothing in the way,
merge it back into trunk as soon as possible.
If you can't merge it back into trunk for whatever reason,
just continuously pull trunk back into your feature branch.
So HL, you've heard Pierre Chen talk
about his version control system for a few minutes.
Did you hear some things where you thought about
and thought, I would really like to have that
in my version control setup?
Or maybe you can talk about a few other improvements
that you would like from your version control system.
You must have been thinking about this in the past years.
I mean, I think every Git user has.
I like Git, but it just needs to add this extra feature.
What Pierre Chen was mentioning, it's really interesting
because the workflow that he described,
it was very, very easy, like a communication between humans
and saying, like, this is what has changed.
And because you don't have so many open contexts,
then it's easier to keep track to what is actually the change.
Because let's face it, as humans,
we can only retain a very small amount of things
in our head, in our working memory.
So going beyond that doesn't make sense.
We can be working in 10 different features,
but not at the same time.
We can't.
That is not human possible.
So at some point, it's what Trish is saying.
You have to reduce it to the very minimum,
work on it, and serialize.
Sadly, for all the synchronous that we like to talk,
we are very serial, like humans.
We focus on this and then on that.
We can have different ideas during the day,
and that can be totally asynchronous and in parallel.
But working, working is something
that we have to start and finish at some point.
So the simplicity of our tools that
allow us to actually focus on a specific thing
makes total sense for me.
Now, the other thing that brings a lot of problems
into development is the human factor.
And I put a link about conventional commits
because my whole point is, I'm not
going to even talk about long-lived feature branches
from, no.
It's like, write down what, not what,
why you have to introduce those changes.
What?
You have the freaking divter.
That's not a question.
I can see what happened, but not why.
Follow a pattern.
I agree with your team.
What makes more sense?
The whole history of Git should give you a more enriching idea
of what was happening, what actually, maybe not
what we were thinking, and we didn't go in that direction,
but at least hinting out, this is why it actually happened.
Not what actually happened, but why it happened.
I think for tools that follow better the human interaction
of how human works, it's going to reduce the overhead for us
using it, and we will be more efficient using them.
So that's why it totally makes sense for me.
Please be kind to your fellow developer
because you may be the one that is going to go,
what were we thinking?
Wait, were we thinking at this point?
I really want to follow up on that as well, the why, not what
on commit messages.
Particularly, again, coming back to IntelliJ IDEA,
I quite often use the Git annotate on the IDE.
So when I'm looking at a line of code and I see,
I don't know, some weird if statement, if I use annotate
and it says, and it has the commit message there in the IDE
and it says, I was working on something
which makes sure that users don't get screwed over
by something, something, something,
you're like, oh, right, now I know why this if statement exists.
So now I know what this test is trying to do.
Whereas if it says, added a test to make sure
issue number 521 works, I'm like, that's not very helpful.
So all what you're saying is,
we should be talking about changes.
And you're actually, when you're talking about commits,
we're talking about changes, yet you're using a system
that only stores snapshots.
So Git is entirely about the what.
A snapshot is an entire version of your project.
So you're entirely about what is there.
You're not talking about what changed since last time.
In order to get that back in Git,
you have to recompute it from two different commits.
And you have an automated machine generated answer
that doesn't, that might make sense, but sometimes doesn't.
Like I have examples of where, of like giant Git failures,
they're obviously not too common,
but merges where Alice and Bob are working in parallel.
Alice is working towards the beginning of the file
and Bob is working towards the end of the file.
And when they merge, Bob's new changes,
so at the end of the file are inserted
into Alice's new changes.
And that's a common, that's an issue in three-way merge
and there's just not enough information to work.
So you could not possibly solve the merge problem
that Git is trying to solve correctly in all cases.
This is just to mean like, yeah, you're talking about
what you want as changes and what everybody wants as changes.
Nobody's ever seen a comment, actually,
except maybe Git developers.
When you are trying to visualize a comment in any UI,
you get presented with a change.
And that's not what a comment is.
And you've probably, since you've done advocacy
for all these years, you've probably seen countless examples
of junior or maybe not so junior devs saying, what?
No, no, that's impossible, a comment is a change.
No, a comment is not a change, it's not a diff.
I think I already encountered some love
for some of the features of Bkool
with the other two guests that we have.
And I think it would really fit well
with this drunk-based development approach
that we have been talking about.
So Pierre Chen, I've been meaning to ask you,
do you understand why so many people
are still using Git currently?
Can you think of a reason why they are?
I can think of many reasons.
One thing I haven't stated since the beginning
of this podcast is that I'm a big Git fan, actually.
I also use the command line and I also look at the ref log
and I love how it's, it's just not meant to merge
because it was designed by people who don't like,
like, they have very strong opinions
on how they collaborate with others, right?
Kernel Linux or Kernel developers.
It's a very exclusive team working in one very rigid way
that is not suitable for all kinds of projects.
It's also, like, the Linux kernel is its own thing, right?
It's probably the most, the farthest outlier
in all software development projects that there is.
So, and we're all taking their development tools
and saying, well, if it fits the Linux kernel,
it probably fits our project,
but no, that's completely wrong.
Yeah, so one thing is, yeah, if it just works,
Git really, really works and it works well.
It's very elegant.
So I used to say simple
because I take that as compliments,
but sometimes people say, well, they see simple
and they're like, oh, simple is probably not a feature,
but like, actually being simple
is actually a really strong feature.
Designing simple systems that work is great.
And so you can get it simple.
Now I prefer to say elegant
because it's less ambiguous, let's say.
So yeah, Git is really elegant.
Its data structures are really, really small
and they don't do much, right?
They don't do their rebates as well.
They don't do their merge as well.
They don't do, they are completely unable to cherry pick.
Even though they try to mimic all these commands,
they're unable to actually implement them
fully, properly, rigorously.
But yeah, it's a great tool.
So it works, it actually works.
Behold is much more complicated.
It's very simple to use, very easy to use,
but it's also extremely complicated
under the hood to implement all, like to make it fast.
Like, for example, I've had to write database backend,
especially for people, so this was a nightmare to write.
Like it's called Santa Kevye,
it means dictionary in Finnish.
It's just a key value store,
which turned out to be the fastest
open source key value store there is,
but it was also a nightmare to write.
And the reason I needed to write it
was not even performance.
The reason was that I want to be able to fork,
to fork a key value table.
So when you have a dictionary,
you want to be able to just fork it
and get two separate copies without copying a single byte.
So that's what Santa Kevye does,
using persistent data structures,
garbage collection, reference counting, and so on.
So, but this is like a giant machine.
It's really, it's really complicated.
Like there's few lines of codes,
but it's not easy.
That's one reason, but I doubt this is the main reason,
because very few people know the internals of Git.
I think one of the reasons is that, well, it works.
People have nightmarish memoirs of switching to Git.
When they switched from SBN
with all these little hacks that they were using,
and they switched to Git,
I know a trivia from some companies
where people actually physically fought
over this issue of switching to Git.
Like during the meeting, two men, obviously,
women wouldn't do that, I guess.
So two men at some point said,
well, let's settle it out just outside the building.
So now they're looking at a new tool with people or whatever,
and they're like, no, we're not going through that again.
So that's one of the issues.
Another issue, and it's actually appeared in the chat,
in our podcast chat here,
the tools aren't quite ready yet.
For example, for Git, we have GitHub.
It hasn't innovated in any way in a number of years,
but it works.
There's 500 developers taking care of your source codes.
For Pughu, we've had some,
well, I wrote something called The Nest.
I released it with the version of Pughu
back in November, 2020, I guess.
It's okay, but it went through a number of issues.
Like for example, the OVH data center fire in March 2021,
while my server was in the fire.
So I've had to rewrite a replication layer
to be able to replicate it in three different data centers.
So I did that because the tools that this uses,
like PostgreSQL, don't really work well
because I want to do, like the main issue in summary
was that I wanted to do a join, like a database join,
between information that's stored in Pughu repositories
and information that's stored in PostgreSQL.
So in PostgreSQL, you can easily do a join between two tables,
but you can actually do a join between two databases,
especially if one of them is not even stored in PostgreSQL,
it's stored in the form of Pughu repositories.
So this doesn't work very well,
and it causes the machines to go down sometimes,
not actually go down, but just be under too much load.
And so this should be okay,
but actually because of how PostgreSQL works,
it just sometimes looses data.
So that's really bad.
Having downtime is okay, losing people's data is not.
So I decided to rewrite this and released it recently,
but we're currently going under a major rewrite
and redesigned the ecosystem around repository hosting.
So for example, just taking these things,
this is probably very scary for lots of teams.
So Pughu itself works and works very well,
and you can use this readily in repositories,
you're very unlikely to encounter bugs.
We're testing it all the time on crazy scenarios
and huge repositories and so on.
So the tool itself is ready,
the tuning around it, like hosting services and so on,
we're currently finding a better way to do it.
It's probably just a matter of weeks or something.
The majority of the ecosystem is not the same.
Yeah, because you mentioned a few possible reasons
why people would still be using Git.
So I wanted to ask you two small questions.
The first one is,
is it actually one of your goals
to grow Pughu's market share?
And secondly, if yes,
what do you think would be needed to achieve that growth?
Well, the initial goal was that the problem was interesting.
And that's how Florian, Becker and myself started working.
It wasn't, we didn't have any idea
that it would result in something like that,
or like, oh yeah, Darks has huge performance problems.
And Florian was very knowledgeable about Darks,
he was one of the maintainers.
He told me, well, no one really knows
why the merge can sometimes take exponential time.
And we're like, oh, actually we're computer scientists
working all the time on asynchronous computing models.
So this is a job for us.
Okay, let's do it.
And so we looked at the bibliography
and started noticing some cool algorithms
that we could use.
And we needed to expand on them and so on.
So this started because the problem was interesting.
Yeah, growing market share.
I think our industry is quite conservative in that regard.
It's hard to make people change.
We like to see ourselves as the heroes
of progress and modernity,
but actually we're very, very conservative
when it comes to our own methods.
This would be growing people's market share
in the current markets of GITS will be hard.
But there's another market
that's completely unreachable by GITS
that's people not even doing code,
legal documents, office documents.
All these people, they desperately need version control
and they cannot possibly use tools like this.
Even if you restricted their files to formats,
we know how to handle a GIT like markdown later
and these things, even that would be really hard to use.
Go teach a parliament member how to rebase
and whether they should merge or rebase
or cherry pick something from another vote.
This is completely impossible.
So same thing for liars.
Liars work really fast basis, much faster than us.
And so they could not possibly spend half a day
dealing with two branches that have gone wild
because one of the junior liars in the team
decided to do a reverse or whatnot.
These markets are definitely addressable by people.
And there's another niche in our industry
which is video game developers.
So these people don't see their needs addressed by anybody.
Most teams are currently using Perforce,
which is SVN was CVS done right
and Perforce is SVN done wrong.
So yeah, most video game teams are still stuck with that.
Sorry, I really wanted to follow up on something
you said there, which is absolutely fascinating to me.
It's difficult to, it's impossible to persuade a lawyer
to spend half a day like fixing up a merge
which has gone wrong.
And this is something which is,
this comes back down to the DPE stuff
that we're talking about at Gradle.
As in the software industry,
we're kind of fine for really expensive software engineers
to spend half a day or a day mucking around with merges.
And these are the sorts of things
that get in the way of our productivity.
And yet we kind of tolerate them.
It's fine to spend a thousand dollars
on the fact that the merge went wrong.
And I think that we have become very tolerant
of this kind of thing precisely
because we don't want the pain of changing tools
or changing or migrating.
And I get that too,
because migration is an enormous mess.
But we will tolerate small pains like half a day here
and a day there on tools and processes
which are not working for us.
And I think that this,
particularly in the development environment,
in the production environment,
we've got a lot better at circling back
to where we started this conversation.
Got a lot better at automation,
pipelines, smoothing things out.
But when it comes to the development environment,
we still tolerate this nonsense of expensive toil.
And I think that's kind of crazy.
Yeah, I think liars would just laugh at us if they knew.
Yeah, it seems to me
that Pyhool could also really benefit then
from plugins for popular IDEs
or maybe graphical ways to show the state
of your version control
because lawyers are not going to learn
Pyhool command line.
I can imagine.
Absolutely, yes.
And it's very,
the people model is very suitable for that.
You can easily visualize patches
and patches is something everybody understands.
And you can also have patches
that are not necessarily on lines,
but it can work on words, for example,
or any unit actually can change
the file program as you please.
Interesting.
So HL, I wanted to ask you a question
that is about 10 years into the future.
Suppose we could fast forward 10 years
into the future right now
and we would be able to compare
that state of source control
and build automation to the current state
that we have right now.
What do you think would be the biggest difference?
Certainly as we have heard,
even though we have, as Pierre mentioned,
a very, very elegant tool right now,
for the command that we have set,
this is really not the best suitable.
Like 18 years,
not all developers know how to use it
to its full potential,
which either points towards
we are difficult to teach
or maybe the tool is not working.
It's like when people say to go against the grain,
maybe this is the case.
So in my point of view,
there are so many things that we have learned
from managing a large amount of information
that changes in a strategic or in a specific way.
Is this the best we can do?
Certainly not.
The numbers speak for themselves,
the experience of the developers speak for themselves.
So I know that it has to be a change.
One of the changes that I have seen,
but it's a totally different thing,
it's you have your Git
and you're building a lot of other tools on top of it.
Block tracking, CICD, documentation, et cetera, et cetera.
So I have heard that,
and I have seen that a lot of people
when they refer to the next set of tools,
they refer to those.
More things to do near the source code.
But that doesn't mean
that it's a different version control system.
You see the difference.
So I think both are going to evolve naturally.
We will have to find a way
to actually make it more interesting for developers
to use efficiently, proficiently and easy,
something that is really important.
I mean, we're talking about people collaborating
in large code bases at the asynchronous times.
And we cannot only rely on best practices
because best practices are amazing,
but it's not a rule.
And we are not very good at not following rules.
So the future, something different
that I hope more developers and even non-developers can use
without going into the merge hell
that we have spoken at so many points
during this conversation.
I think a lot of other tools will leave
very, very close to source because we have seen it.
We need CICD, we need documentation, we need,
even I think testing will, because it's so important,
it will leave in a different way.
Right now being part of the source code,
it's super important, it's really important.
But now we have more integrations
and suddenly you start seeing like,
developers don't know how, where to put them
or like to put them.
I, for example, consume an integration test consumer
from the producer perspective.
You already have problems like locating it in some way.
So I think it will actually live in its own close,
but in a different way.
That's what I hope.
Something that works efficient for the computer,
that computers can work with it
in the most automatic way,
but also developers can work in the human way too.
And Tricia, what do you think 10 years from now,
the world of source control and build automation?
I think Ishael's right.
I hadn't even realized it was ripe for change
until we had this conversation, but it's ripe for change.
I think there's a number of things
that we've talked about today, which are really important.
For example, we've seen that the Git is very fully featured.
Simple-ish, like it's implemented in a simple way,
but unfortunately exposes itself in a complicated way.
So it means that developers have to know a whole bunch
of different ways to work with it
and then figure out their own human best practices
to layer over the top.
Or like Ishael was saying, that maybe having tooling on top
to make it simpler to work.
I really like what Piotr said about,
what if you have something which is simple to work with
and complicated underneath?
I mean, that feels a little bit like Java,
the programming language, right?
From a programming language perspective,
it hasn't evolved that much,
but it's kind of simple to understand.
But the JVM does a lot of hard work for you
and you don't have to worry about it.
That's why it's one of the fastest languages in the world.
And you don't see that and you don't have to code for that.
And that's where I would like to see
our version control systems go.
So you use it, it just works,
and maybe it can evolve in the background
without you having to worry about new commands,
new ways to work with it, new best practices.
So I think something which is a fundamental building block
that we don't have to care about
helps develop a productivity
because you can automate that stuff and not worry about it.
And then we can worry about the things we care about.
I think layering on process and people stuff
over the top of something to fix the holes in our tooling
is not sustainable, it's not scalable.
We can't keep patching around these problems by having,
for example, code reviews and pull requests.
I've seen a lot of complaints about how,
what if you have a process
that doesn't have code reviews or pull requests?
How will you be able to function as a team?
Well, what if we have things like pair programming,
which I prefer?
Or what if we actually invested in our juniors
so we could rely on them to write code
instead of assuming they write terrible code
and having to do tedious code reviews at the end of it?
These kinds of things.
What I really liked about what Pierre Etienne said as well
is this idea of as developers in particular,
we like to solve problems.
So we see a problem and we solve it
by perhaps layering something over the top of it.
But what if we could step back and go,
what if we can eliminate this problem?
What if this problem is a side effect of,
what if it's a smell of an incorrect tool
of something which can now be fixed
by a different way forward?
So I think that papering over these problems
with processes or extra tooling,
it might be better to step back and figure out actually
what should we be doing instead?
The other thing I wanted to say is we haven't mentioned AI
at all during this conversation, right?
And in 10 years time,
I think AI is going to be a much larger part
of the developer tool set.
I don't know what it looks like yet.
I don't think it's pairing with chat GBT,
but you know, who knows?
I wanna kind of circle back to this idea
of let's not use AI to paper over problems,
for example.
Let's not just use it to go,
well, maybe AI can do our code reviews.
Maybe AI can do our merges.
Okay, but what if we didn't need to do those things?
What if there was a better, faster, easier, cleaner way?
So we didn't have to keep layering on more tools
and more technologies.
So I think that's the only thing
I really wanna say about AI.
I think it will probably contribute to this space.
I would like us to not just add it as an extra piece
of noise for us to debug later on down the line.
Richard, I was so looking forward to talking with you
for a morning without having to mention AI once.
No, I don't blame you at all.
Of course, that's a major influence
and I tend to agree with you.
Pierre Chen, you have the privilege
to utter the final words of our conversation.
What do you think of the world of build automation
and source control in 10 years?
Anything you would like to add?
Lots of things and nothing at the same time.
You know, as Yagi Vera said,
predictions are hard, especially about the future.
In this case, I think we've seen
a dramatic acceleration in the pace of change
that in technologies were used.
For example, if you think about not 10 years from now
but 10 years ago, Docker was just the beginning.
There was no goal.
Scala was starting like we were talking
with Java developers here.
So Scala was a thing but not as widely used as today.
There was no, like MixOS was absolutely unusable
and there is no Angular React in the JavaScript world.
There was no, like Web 2.0 was a thing
but not actually a thing.
So, and then we did it at TensorFlow,
for example, for machine learning.
If you look at all these,
and I'm probably forgetting lots of stuff, right?
If you look at all, like yeah, Rust, Rust was a prototype
from Mozilla aiming at like slightly better C++.
Yeah, predictions, like things have been accelerating
dramatically in the last like 10 to five years.
And so I don't know what the future will hold.
It's really hard to tell.
If I were to give like a tiny prediction
about version control only,
because it's a topic I know quite a bit.
Yeah, I think people will probably,
maybe some teams will switch to something like Gihul
or maybe some, maybe not Gihul itself,
but maybe like the workflows we're allowing people to do.
I think this will be used elsewhere.
We're already seeing the influence
of the things we've been doing in other products.
Like even at GitHub and GitLab,
like people are using like the experimental things
we've allowed ourselves to do.
They've been used in other products.
Like for example, if you look at the comments
signed by defaults in GitHub,
that's something that's with it first.
And well, discussions instead of full requests.
So people as a merger between discussions,
like in the nest,
we're using a completely different model from full requests.
We're using just discussions
and you can attach batches of discussions.
They are not full requests or merge requests.
They're just, yeah, you can start discussing on the topic
and then push it back and then say,
oh, I actually have a solution to that.
Like we should patch the discussion.
Then you can review the batch independent of its context
and just apply it as you want to your main branch
in your computer, because you can always undo it.
You can apply it to different contexts
so that you can test it like that.
So we're already seeing the influence of the stuff
we've done in other products.
GitHub has introduced discussions,
signed commits by default, this kind of thing.
So yeah, I don't know the future, we'll see.
That's always the best answer, I think.
We can all agree that it's a very interesting field
that is changing rapidly.
And I'm planning to be there every step of the way
because I really like where the field is heading.
I had a lot of fun.
Thanks, special thanks to our guests,
Tricia, HL and Pire Chen for sharing your thoughts
on version control on build automation.
And of course, we want to thank you
for listening to this podcast.
Keep an eye on FooJ for future articles about development
and everything related to the Java world.
Thank you and until next time.
Give me a Foo, give me a J,
give me the friends of OpenJDK.

*Based heavily on fetsorn's draft spec*

## Project goals
User-friendly:
    - Fast
    - Clear error messages
    - Mental model maps to implementation (as opposed to e.g. git)
    - Well-documented
    - Easy to install
    - Consistent interface (terminology, design etc)
    - Support internationalization
Robust:
    - Theoretically sound
    - Natively supports conflicts
    - Supports different workflows (monorepo, multirepo, etc)
## Spec
Tooling:
    - Machine-readable output for every command
    - Well-documented process of integration
    - Clear APIs that cover common application use-cases
Repositories:
    - Initializing a repository:
      - Creates all required files/metadata (e.g. `.pijul`)
      - Generates extra files like `.ignore` with useful defaults
    - Cloning a repository:
      - Multiple protocols: ssh/https/etc
      - Downloads all required files from remote
      - Creates a new repository with exactly same changes as remote
Identities:
    - Easy to create
    - Multiple identities
    - Modifying details such as name/email/etc
    - Security considerations such as passphrases, rotating keys, revoking keys, etc
Remotes:
    - Easy to add/edit/remove remotes
    - Seamless, well-documented way to self-host over ssh/https/etc
    - Per-remote patch sets; remote A can have different patches to B for the same repo
    - Push/pull over ssh/https/etc
Changes:
    - Find modifications to files not already recorded in a change
    - No staging area, working dir is recorded directly
    - Select patches to record change (can select only some changes to a single file)
    - Split changes into multiple smaller changes
    - Amend previous changes with new metadata/patches
    - Sign changes using identity
Sets of changes:
    - Compare sets of changes against each other
    - Compare single change to set of changes
    - Compress set of changes into snapshot ("tag")
    - Handle merging two sets of changes
Files:
    - Track/untrack files
    - Add entire directories recursively
    - Handle renames, moves, deletes, etc
    - Purge file from history (`git filter-repo`)
    - List files that are tracked/untracked
    - Ignore files user has requested to ignore (via `.ignore` or similar)
CLI:
    - View status/summary of entire repository
    - Automatic internationalization to system language (e.g. french users get french text)
    - Consistent CLI terms & idioms
    - View log of all changes
    - View log of most recent change
    - View modification history of changes (e.g. change X created, amended to include added file, unrecorded)
    - Compare differences with remote before completing `push`/`pull` operations
Maintainers:
    - Download user-submitted changes without polluting work
    - Amend (or at least propose amendments) to title/description without needing author input?
    - Add author credit to changes not signed by real author
# Pijul requirements
same list written as "user must/should/could won't have"
## Project goals
User-friendly:
    - user must feel that pijul is fast
    - user must see clear error messages
    - user must have mental model maps to implementation 
    - user won't use implementation that has no mental model map (e.g. git)
    - user must read good documentation
    - user must feel it's easy to install
    - user must feel interface is consistent
    - user must feel terminology is consistent
    - user must feel design is consistent
    - user must read interface in i18n languages
Robust:
    - user must read theoretically sound writeup
    - user must resolve conflicts natively
    - user must use monorepo workflow
    - user must use multirepo workflow
## Spec
Tooling:
    - user must get machine-readable output for every command
    - user must integrate // ? what is specific integration example here
    - user must read good documentation for the process of integration
    - user must feel that API is cleaer 
    - user must see API for common application use-cases // ? what are specific common use cases here
Repositories:
    - user must initialize a repository
      - user must create all required files/metadata (e.g. `.pijul`) // ? what are all specific required files here
      - user should generates extra files like `.ignore` with useful defaults // ? what are all specific extra files here
    - user must clone a repository
        - user must clone a repository over ssh
        - user must clone a repository over https
        // ? what other specific protocols here 
        - user must download all required files from remote // ? what specific required files are here
        - user must create a new repository with exactly same changes as remote // ? what specific cases here
Identities:
    - user must create identity
    - user must feel it's easy to create identity
    - user must manage multiple identities
    - user must modify name
    - user must modify email 
    // ? what other specific identity details are here?
    - user must feel it's secure to use an identity
    - user must use passphrases to secure identity 
    - user must rotate key 
    - user must revoke key
    // ? what other specific security cases are here?
Remotes:
    - user must add a remote
    - user must edit a remote
    - user must remove a remove
    - user must feel it's easy to manage remote
    - user must read good documentation 
    - user must self-host over ssh
    - user must self-host over https
    // ? what other specific ways to self-host
    - user must have multiple per-remote patch sets // ? what simpler way to say this
    // remote A can have different patches to B for the same repo
    - user must push over ssh
    - user must push over https
    - user must pull over ssh
    - user must pull over https
    // ? what other push/pull protocols 
Changes:
    - user must find modifications to files not already recorded in a change // ? what simpler way to say this
    - user must learn that working dir is recorded directly, there's no staging area 
    - user must pick which patches for each file to record change
    - user must split changes into multiple smaller changes
    - user must amend previous changes with new metadata
    - user must amend previous changes with new patches
    - user must sign changes with identity
Sets of changes:
    - user must compare sets of changes against each other
    - user must compare single change to set of changes
    - user must tag, compress set of changes into snapshot 
    - user must merge two sets of changes
Files:
    - user must track file
    - user must untrack file
    - user must add a directory recursively
    - user must rename a file
    - user must move a file 
    - user must delete a file
    - user must purge file from history (`git filter-repo`)
    - user must list tracked files
    - user must list untracked files
    - user must list tracked status of all files
    - user must ignore a file user via `.ignore` // ? what other specific "similar" ways to ignore
CLI:
    - user must view status/summary of repository
    - user must use cli in system language according to i18n
        - french user must see french text in cli 
    - user must feel the CLI terminology and idioms are consistent
    - user must view log of all changes
    - user must view log of most recent change
    - user must view modification history of changes
    - user must see when a change is created
    - user must see when a change amended to include added file
    - user must see when a change is unrecorded
    - user must compare differences with remote 
       - before completing `push`/`pull` operations // ? why before, why not always, what happens if this fails, why this needed
Maintainers:
    - maintainer must download user-submitted changes without polluting work // ? how is work polluted
    - maintainer must see clean work after downloading user-submitted changes // ? how is work clean
    - maintainer must propose amendment to title and description without author input
    - maintainer should amend title and description without author input
    - maintainer should add author credit to changes not signed by original author

## User-friendly
 - user must feel that pijul is fast
 - user must see clear error messages
 - user must have mental model map of .pijul implementation 
 - user must read good documentation
 - user must feel it's easy to install
 - user must feel interface is consistent
 - user must feel terminology is consistent
 - user must feel design is consistent
 - user must read interface in i18n languages
## Robust
 - user must read theoretically sound writeup
 - user must resolve conflicts natively
 - user must use monorepo workflow
 - user must use multirepo workflow
## Tooling
 - user must get machine-readable output for every command
 - user can integrate pijul into a command line server like soft-serve
 - user can track issues alongside alongside the code like fossil
 - user must read good documentation for the process of integration
 - user must feel that API is clear 
 - user must see clear API for cloning
 - user must see clear API for remote management
 - user must see clear API for identity management
 - user must see clear API for conflict resolution
 - user must see clear API for change history
## Repositories
 - user must initialize a repository
 - user must see automatically generated .pijul directory
 - user must see automatically generated .ignore file
 - user must see useful defaults in generated .ignore file
 - user must see .DS_Store in generated .ignore file on MacOS
 - user must see .git in generated .ignore file if directory has .git
 - user must clone a repository
 - user must clone a repository over ssh
 - user must clone a repository over https
 - user must see new directory with repository name
 - user must download project files from remote
 - user must download .pijul from remote
 - user must download .ignore from remote
 - user must import .git repository to .pijul repository
## Identities
 - user must create identity
 - user must feel it's easy to create identity
 - user must manage multiple identities
 - user must modify name
 - user must modify email 
 - user must feel it's secure to use an identity
 - user must use passphrases to secure identity 
 - user must rotate key 
 - user must revoke key
## Remotes
 - user must add a remote
 - user must edit a remote
 - user must remove a remove
 - user must feel it's easy to manage remote
 - user must read good documentation 
 - user must self-host over ssh
 - user must self-host over https
 - user must have different patch sets on remote A and remote B
 - user must push over ssh
 - user must push over https
 - user must pull over ssh
 - user must pull over https
## Changes
 - user must see modifications that are not yet recorded in a change
 - user must learn that working dir is recorded directly, there's no staging area 
 - user must pick which patches for each file to record change
 - user must split changes into multiple smaller changes
 - user must amend previous changes with new metadata
 - user must amend previous changes with new patches
 - user must sign changes with identity
## Sets of changes
 - user must compare sets of changes against each other
 - user must compare single change to set of changes
 - user must tag, compress set of changes into snapshot 
 - user must merge two sets of changes
## Files
 - user must track file
 - user must untrack file
 - user must add a directory recursively
 - user must see how many files were tracked
 - user must see how many new changes are tracked
 - user won't see notification "tracked 1 path" if the file is already tracked
 - user must rename a file
 - user must move a file 
 - user must delete a file
 - user must purge file from history like `git filter-repo`
 - user must list tracked files
 - user must list untracked files
 - user must list tracked status of all files
 - user must ignore a file user via `.ignore`
## CLI
 - user must view status/summary of repository
 - user must use cli in system language according to 18n
 - french user must see french text in cli
 - user must feel the CLI terminology and idioms are consistent
 - user must view log of all changes
 - user must view log of most recent change
 - user must view modification history of changes
 - user must see when a change is created
 - user must see when a change amended to include added file
 - user must see when a change is unrecorded
 - user must compare differences with remote 
## Maintainers
 - maintainer must download user-submitted changes 
 - maintainer must be able to edit directory after downloading user-submitted changes 
 - maintainer must propose amendment to title and description without author input
 - maintainer should amend title and description without author input
 - maintainer should add author credit to changes not signed by original author