Project
TeamCity
Priority
Major
Type
Feature
State
Open
Assignee
Pavel Sher
Subsystem
Server
Affected versions
No Affected versions
Fix versions
High Priority Pool
Fixed in build
No Fixed in build
Fixed in builds
no build yet
  • Created by   Yegor Yarko
    13 months ago (06 Apr 2011 16:33)
  • Updated by   Yegor Yarko
    2 weeks ago (09 May 2012 14:05)
 
TW-16052 Feature branch support for DVCS
46
Issue is visible to: All Users
  The issue is visible to the selected user group only
You don't have permissions to view 1 more attachment to this issue
Applicable for git, mercurial and other DVCes.

When a team uses feature branching, it gets tiresome to manage that in TeamCity as a new build configuration needs to be created for every new branch.

The goal is to have single set of build configuration settings but allow it to build many branches.

Suggestion is to:
  • in build configuration VCS settings, allow to specify branches to be monitored (a pattern or set of patterns)
  • detect changes in all the matching branches, display the changes considering their parents (a tree structure) - probably as most git/hg history displaying tools with list of changes and branches linking as vertical lines to the left, allow to set current branch (branches) in a build configuration to display only changes of the branches selected
  • trigger builds on changes (according to VCS trigger settings)
  • display the builds in hierarchy defined by changes hierarchy, mark builds with "tags" for their branches. Allow to see all builds and only builds from single/selected branches. Appropriately determine new/not new status of the tests/problems, send appropriate notifications
  • allow to see/get notifications for the builds in "my branches" - branches I've committed changes into.

In this case, status of build configuration is something not very valuable (it would mean all branches build OK), probably it should be divided in presentation to have separate status for every branch in the build configuration.

TBD: in case of several VCS roots, how changes hierarchy should apply to builds hierarchy
Comments (31)
 
History
 
Linked Issues (?)
 
Patrick Linskey
  Patrick Linskey
08 Jun 2011 04:47
11 months ago
The other thing I'd add to this list is:

  • allow activities to have conditional execution, based on branch name.

As a first-order approximation, if I have changes in 'master', after tests are complete, I'll want those changes to be moved along to the next step in my build pipeline. If I push changes to 'pcl-remote-tests', I do not want the test completion to trigger the next build pipeline step.

I can imagine at least three implementation approaches:

1. allow a build step in a configuration to be executed only if a certain branch regex matches
2. create a new configuration phase called 'after-tests-complete' or something, and allow that to be configured to only run if a certain branch regex matches
3. move the downstream work to a separate configuration that is triggered by the successful completion of my tests, and allow that configuration to also have a branch-regex-based trigger. In this case, I'd want to use a snapshot dependency probably.

Personally, I'd like to be able to do that sort of work within a single configuration, as it simplifies the agent provisioning story etc. vs. running in multiple configurations.

Also, I can imagine making this more complex (I think that some of the linked issues delve into these details). For example, if I push changes to 'pcl-remote-tests', I might want to push that to master automatically. If master has no new changes vs. what's in pcl-remote-tests, then it'd be safe to just go ahead and perform the master-oriented deployment work.
Michael Gerasimov
  Michael Gerasimov
14 Jun 2011 13:56
11 months ago
1. Feature Branches как стиль разработки. На данный момент эта возможность vcs'a становится полномасштабной частью нашего процесса, который подразумевает параллельную разработку нескольких версий продукта, в каждой из которых идёт независимая разработка отдельных фич. Такой стиль даёт возможность устойчивого тестирования выделенных версий и предсказуемой оценки качества и сроков релизных кандидатов. Не буду здесь рисовать полную картину параллельных бранчей, но она уже сегодня подразумевает одновременное сосуществование 7-ми веток только в .Trace'e.

2. Ветки создаются отдельными девелоперами по необходимости, здесь нет централизованного требования, за исключением того, что только те ветки, которые полезны всем должны находить своё отражение в публичных конфигурациях ТС. То есть личные экспериментальные ветки в ТС попадать не должны. Необходимость публичности бранчей в ТС объясняется ещё одной методологической особенностью нашего процесса - каждый член команды может взять подзадачу из любой разрабатываемой ветки любого продукта, чтобы её продолжить. Таким образом, публичность результатов, в частности тестов, является необходимостью.

3. Ветки могут жить от нескольких дней до нескольких месяцев. Как ясно из п.2 вопрос об "интересности" бранча для отд. разработчиков не стоит, поэтому ограничение видимости, на мой взгляд будет только вредить.

4. Интеграция изменений между бранчами. Не очень понятно, какое отношение имеет этот вопрос к ТС. На сегодня, интеграция (merge) осуществляется средствами Mercurial per se и визуализируется с помощью, например, Hg Workbench. Факт мержа отражается, в том числе, в комментариях к коммитам. Поэтому, для ТС эти мержи, по идее, не должны ничем отличаться от обычных коммитов, если только вы не хотите рисовать параллельное дерево с Hg (может, это и не плохо).

5. Закрытая ветка - это закрытая ветка. Моё личное мнение - соответствующая build-конфигурация должна быть удалена, возможно не сразу. Возможно, после закрытия она должна быть видима только автору брэнча. Ветки ведь закрываются не просто так, их результат сливается в другую ветку (не обязательно транк), после чего соответствующая функциональность живёт совместной жизнью с другими. Рабочие ветки вряд ли метятся тагами, если только для собственных нужд, в основном, у нас сейчас принята практика отмечания (пред)релизных билдов в QA-ветке. В конце концов вся информация остаётся доступной в репозитории, и, если кто-то захочет порыться в "анналах", всегда можно локально сапгрейдиться (во блин, слово...) на нужную версию закрытой ветки и поиграться.

6. Возможность настраивать нотификации по билдам. Да, вот нотификации, мне кажется, не должны приходить всем. Только тем, кто в них сейчас заинтересован.

7. Взаимосвязь между настройками базовой конфигурации и её потомками для других бранчей мне кажется самым мутным вопросом. На первый взгляд - безусловно настройки должны наследоваться и синхронизироваться. С другой стороны, не могу привести обратный пример. Я думаю, этот момент мне надо обговорить с коллегами, чтобы прийти к более конкретному мнению.
Michael Kuzmin
  Michael Kuzmin
16 Jun 2011 18:48
11 months ago
feedback from http://devnet.jetbrains.net/message/5306501
In our workflow, our developers often collaborate on shortlived feature branches (ie, <2weeks). 1-2 developers + 1 tester. Using personal branches worked nicely for this, avoiding creating new configurations for all short branches ; but makes it tricky for the tester to locate new builds.
Clifford Hammerschmidt
  Clifford Hammerschmidt
08 Jul 2011 01:13
10 months ago
+1!

if the "personal" option supported a "public" mode checkbox we'd be 90% of the way there by using "/refs/heads/project-*".
Chris Nicola
  Chris Nicola
09 Jul 2011 01:39
10 months ago
Would love to see this as well, a lot of git-based teams follow the git-flow workflow and so getting build results on feature branches which are close to the "develop" branch would be valuable.
http://nvie.com/posts/a-successful-git-branching-model/

In git-flow it is common for features to be "feature/some-feature-name", hotfix branches are "hotfix/X.Y.Z" and release branches are "release/X.Y"
Yegor Yarko
  Yegor Yarko
24 Aug 2011 17:19
9 months ago
a bit related comment in the forum: http://devnet.jetbrains.net/message/5317378#5317378
Peter Mounce
  Peter Mounce
09 Nov 2011 01:58
6 months ago
+1. I'd want the ability to assign different branches into agent priority classes, so that for example builds against master get a higher priority than builds against branches/turn-the-text-blue-please.
Jonathon Rossi
  Jonathon Rossi
26 Nov 2011 09:36
5 months ago
I'd like to see what you guys come up with to provide built in support for this scenario.

Last week I just got annoyed with having to manually create build configurations and reminding people to delete their obsolete configurations, I just wrote an app that syncs our git development branches with a TeamCity project. I used the REST API to read details from TeamCity and had to just POST to the URLs that the dashboard uses to create and delete configurations because the REST API doesn't expose that functionality. It seems to work well enough for us at the moment, but I would be happy to move over to an inbuilt and supported feature.
Jim
  Jim
26 Nov 2011 11:45
5 months ago
Jonathon, this sounds interesting, any chance you could share the code or give some hints how to accomplish this? REST is alien to me, what are you using to accomplish this?
Peter Mounce
  Peter Mounce
26 Nov 2011 17:18
5 months ago
Jonathon, I'd love to see that, too.
John Morales
  John Morales
26 Nov 2011 19:28
5 months ago
We've did a similiar approach but instead of crafting our own POSTs, we used webdriver to automate firefox which would sync teamcity using the branches from a 'git ls-remote' creating and deleting branches as needed based on a main project template. It seems to work will, but like Jonathon, we would rather switch to a supported feature.
Jonathon Rossi
  Jonathon Rossi
27 Nov 2011 19:29
5 months ago
Sorry for the delay, I just posted about what we are doing and a copy of the code:
http://jonorossi.com/blog/archive/2011/11/28/sync-teamcity-build-configurations-with-git-branches.aspx

I apologise to the JetBrains staff if we are going too off topic, so if you want to discuss the code please use my blog or the gist comments.
Yegor Yarko
  Yegor Yarko
28 Nov 2011 16:59
5 months ago
Jonathon, thank you for sharing your solution. It seems to be an "external" solution to TW-7869.

What we want to do in the scope of this issue is to pursue with the approach outlined in the issue description: allow to build different branches in the scope of a single build configuration. With this approach it would probably make sense to have a single development build for all branches and separate one for master to run more builds (like deployment) on it's base.

Dear voters, can you please look through below and comment which one of the approaches you would consider better for you?

Rough approaches comparison:
Single Build Configuration A build configuration per branch
changing a setting in a build configuration is immediately applied to all branches, no way to change TeamCity settings for a single branch can easily change settings for a single branch, can only change settings for all if they use single template
history of builds in branches is preserved (usual build cleanup rules are used)branch builds are deleted when the branch no longer exists in the repository
new "tag"-based filtering is used throughout TeamCity UI (history, current state, notifications, API, etc.)no new entities: separate build configurations

also, there are points which will probably be easier to add in single build configuration approach:
  • display for changes from all branches in a single view, with graph
  • display of builds for all branches in a single view
Eduard Kibort
  Eduard Kibort
28 Nov 2011 20:14
5 months ago
For me would be enough "Single Build Configuration"
Matt Florence
  Matt Florence
28 Nov 2011 20:27
5 months ago
Single Build Configuration
Christian Nelson
  Christian Nelson
28 Nov 2011 20:39
5 months ago
@yegor in the Single Build Configuration scenario, would there be a way to override an env variable for an individual branch build? Multiple branch builds might clobber one another if the test suite uses a database. In the past, I've used env vars to give each branch a unique database instance.
John Morales
  John Morales
28 Nov 2011 21:08
5 months ago
Build configuration per branch allows us to support different triggers for branches, for instance we have git commit triggers on our master and integration branch while feature branches are on demand, all sharing the same build template. I personally like the separation of build configuration per branch sharing a template, but if the same could be reached with tag filters, that could work. But I don't see the real benefit of Single Configuration over Config per Branch, it appears that you lose the ability to change settings per branch without gaining much.
Clifford Hammerschmidt
  Clifford Hammerschmidt
28 Nov 2011 23:18
5 months ago
"single build solution"

Currently I use a script to add/remove project definitions to the configuration file (using a small tool: https://gist.github.com/1401596) that sets the project name (the branch name) that the template they are based on uses to determine what branch to sync for the build. I have this run every 5 minutes and use 'git ls-remote' to determine the branches that should be present (piped into a file the tool reads).
Clifford Hammerschmidt
  Clifford Hammerschmidt
28 Nov 2011 23:41
5 months ago
I think it depends on the model you're using. In the model I'm using the branches are feature branches that are destined for integration into the project branch, so building them with an alternate configuration wouldn't make sense, since ultimately they will be built with the project branches configuration. For things that might 'clobber' each other (db, subdomain naming, etc.) we use the branch name to partition out the resources and pass in the branch name to the build scripts. In my case we have the build deploy to a sub folder, the branch name, and every build is run side-by-side so the testing system can test them independently.
Josh Faust
  Josh Faust
29 Nov 2011 02:24
5 months ago
Build configuration per branch, created based on a template – combined with the new ability to disable build steps and triggers, this is the most powerful I think.
Jonathon Rossi
  Jonathon Rossi
29 Nov 2011 15:19
5 months ago
Build configuration per branch based on a template.

We set up different build triggers on branches and need to be able to modify the build configuration parameters from the template for specific branches. I would also expect our users would like to be able to configure notifications only for their branches and configure their dashboard the same way.
Yegor Yarko
  Yegor Yarko
30 Nov 2011 00:58
5 months ago
Christian,

in the Single Build Configuration scenario, would there be a way to override an env variable for an individual branch build?

Most probably no (at least for a start). However, you will be able to get current branch name inside the build.
Clifford mentions the same approach above.

Is this a critical obstacle for your case?
Yegor Yarko
  Yegor Yarko
30 Nov 2011 01:07
5 months ago
Thank you for the comments!

As to "need to have some branches builds customized":

If you need to change some settings for specific branches, then that means you start to manage them manually and there is no (at least obvious) way to generate those build configurations automatically.

The most common case for "Single Build Configuration" approach seems to include separate build configuration for "master" branch with specific settings and separate build configuration for "all the other feature branches" that do not need any manual tweaking. If you need a manual setting for some branches you can copy a build configuration, set branch name and do change the settings - and this copying logic is _already_ possible.

Are there cases when you need automatic builds for newly appearing branches, but still need to change settings for some of them?
Clifford Hammerschmidt
  Clifford Hammerschmidt
30 Nov 2011 02:09
5 months ago
note: I've been assuming that the build settings in TC are just a hook to your own build system, since otherwise you'd have to merge-up build settings in TC in addition to merging up the code in the branch whenever a branch was accepted into a project. So the branch build settings and the project build settings in TC don't change, but what the build does could change, since that is dependant on the build files in the branch, and not the settings in TC.

Hope that makes it more clear.
Na'Tosha Bard
  Na'Tosha Bard
05 Mar 2012 14:14
2 months ago
Please consider the case outlined in TW-789. If you have multiple build configurations per project (we have about 70 because we have about 70 individual targets that can be built from our, albeit large, codebase), then you need a way to manage projects that are copies of each other and update them in bulk. Having one project for one branch in this case is very nice because it's very easy to browse history on a per-branch basis. So having one build configuration per branch (in one project) simply won't work; it won't feasible to manage one project with 70 build configurations and 50 branches, for example.

It's also not safe to assume any particular branching convention, especially with Mercurial (Mercurial users may be using Bookmarks, or Repository-based Branching instead of named branches, even though named branches are the only branching model really well supported right now).
Clifford Hammerschmidt
  Clifford Hammerschmidt
05 Mar 2012 21:51
2 months ago
Na'Tosha,

Are you storing your build configurations in TC? I'm suggesting that TC should know very little about the differences in the builds and the branches. I setup a script to add new branches based on a template. The template does:

1. sync the code (from branch, repo, bookmark, whatever)
2. set BRANCH_NAME= in the environment
3. run "build.bat" in the sync'd code to build all targets/configurations that are valid for the build agent's environment

The logic for building the branch lives in the branch, as does the code. That logic changes when the code changes require it.

How you determine what a "branch" is (a repo, bookmark, etc.) won't affect the usage pattern. What's important is the code changes that require a change to the build system are paired with the changes to the build system.

So if you have 100 targets/configurations and 100 branches off of master the only branches with different build logic are those that have added code that need changes to the build logic. Most branches will have identical build scripts (or ones with minor changes from that of the parent branch), and the TC template for all of them is the same.

This request (TW-16052) is about building in support for this usage pattern (building branches without having to manually set them up as a new build) into TC directly so we don't have to write an external script to list the branches in the VCS and update the project configuration files to reflect them. (Which is what I ended up doing anyways...)
Na'Tosha Bard
  Na'Tosha Bard
06 Mar 2012 21:34
2 months ago
I'm not sure what you mean by "storing your build configurations" in TeamCity. A TeamCity project is made up of build configurations – in our case, there are 70 per project because there are about 70 different targets that can built from our codebase, many of them depending one each other. My point was simply that if you have many build targets, it's much nicer to be able to easily manage multiple copies of the same TeamCity project. This is currently very difficult. A use case:

---

There is one TeamCity project (call it 'Trunk') that always builds the 'mainline' codebase. Branches are made off of this mainline for each feature. Bob, Alice, and Eve all have feature branches. Each one of of them has a TeamCity project that has the same build configurations as the Trunk project, but the VCS root points to his/her branch.

Eve finishes some work that makes 3 of the build configurations have changed snapshot and artifact dependencies, and adds a new build configuration. She merges her branch into 'maneline' in the VCS, updates the TeamCity Trunk project with the correct artifact/snapshot dependency changes from her own TeamCity project (by hand, because there is no automatic way to do this, which is very cumbersome), and retires her branch, both in the VCS and TeamCity.

Alice and Bob both merge Trunk into their branches. However, they now have Eve's code changes, so they now need the changes that Eve made to Trunk's build configurations to use different artifact/snapshot dependencies. So they have to also make these changes manually in their own TeamCity projects, by hand, becuase there is no automatic way to do it, which is very cumbersome. They also have the option of deleting their TeamCity projects and creating new ones, but they lose the build history of possibly several weeks of work.

---

Now imagine the team is not just made up of Alice, Bob, and Eve, but instead of 75 developers, with 50 active branches, and every TeamCity project has 70+ build configurations.

Our team hits this scenario all the time and it's driving us crazy. I am now in the process of writing a tool to let us manipulate TeamCity projects through TeamCity's REST API that shipped with 7.0, but I don't feel like I should have to.

Regarding assumptions about branching, my comment was about the current assumption that Mercurial users are using named branches for branching. In Git, this pretty much a universally safe assumption, but it isn't with Mercurial – yet the remote build feature in TeamCity for Mercurial and Git assume you're willing to push to a named branch and VCS roots for Mercurial don't supply an option to support bookmarks, only named branches.
Clifford Hammerschmidt
  Clifford Hammerschmidt
06 Mar 2012 22:00
2 months ago
I had similar issues to what you're describing, which is why I have TeamCity use a generic template for projects and call out to logic in the source code branch to execute the building and publishing rather than trying to manually maintain the logic in TeamCity project definitions.

I sounds like you're using TeamCity to manage artifact dependencies between products, which I've avoided doing so far.

W.r.t. this feature request, I'm not clear if you're arguing for it (the ability to build multiple check outs from different "branches" of a product using one TC build configuration), against it, or for something more than it.
Na'Tosha Bard
  Na'Tosha Bard
06 Mar 2012 22:56
2 months ago
Well, we use TeamCity to do things like build installers – which in turn need to build the stuff that goes into the installer. We certainly aren't doing anything revolutionary.

W.r.t this feature request, I'm arguing that it's not enough to do what's in the feature request and call it "support for feature branches in DVCS". From the original feature request: "The goal is to have single set of build configuration settings but allow it to build many branches". This may work for simple cases, but it's not a global solution to managing branches in DVCS.

Sometimes build configurations need to change slightly based on changes made in the branch (maybe you tweaked a dependency list for an installer? Maybe you upgraded the .NET framework requirement and need to specify it run on a certain set of agents?)

Another issue with this approach is that it's valuable to be able to see only the build history for branch X when you have a long-lived branch, which conflicts with the idea of having one project build multiple branches (though this could be solved by adding a "filter by VCS root" or "filter by branch" option in this history view.
Harry McIntyre
  Harry McIntyre
16 Mar 2012 18:08
2 months ago
Those differences should probably be under source control. Maybe the custom TC config for each branch should be stored along with the branch.
Na'Tosha Bard
  Na'Tosha Bard
16 Mar 2012 18:11
2 months ago
Sure, but how do I merge the changes together? Merging the XML files won't result in something usable; IIRC, build configs always need unique IDs . . .