Applicable for git, mercurial and other DVCes.
When a team uses feature branching, it gets tiresome to manage that in TeamCity as a new build configuration needs to be created for every new branch.
The goal is to have single set of build configuration settings but allow it to build many branches.
Suggestion is to:
- in build configuration VCS settings, allow to specify branches to be monitored (a pattern or set of patterns)
- detect changes in all the matching branches, display the changes considering their parents (a tree structure) - probably as most git/hg history displaying tools with list of changes and branches linking as vertical lines to the left, allow to set current branch (branches) in a build configuration to display only changes of the branches selected
- trigger builds on changes (according to VCS trigger settings)
- display the builds in hierarchy defined by changes hierarchy, mark builds with "tags" for their branches. Allow to see all builds and only builds from single/selected branches. Appropriately determine new/not new status of the tests/problems, send appropriate notifications
- allow to see/get notifications for the builds in "my branches" - branches I've committed changes into.
In this case, status of build configuration is something not very valuable (it would mean all branches build OK), probably it should be divided in presentation to have separate status for every branch in the build configuration.
TBD: in case of several VCS roots, how changes hierarchy should apply to builds hierarchy
As a first-order approximation, if I have changes in 'master', after tests are complete, I'll want those changes to be moved along to the next step in my build pipeline. If I push changes to 'pcl-remote-tests', I do not want the test completion to trigger the next build pipeline step.
I can imagine at least three implementation approaches:
1. allow a build step in a configuration to be executed only if a certain branch regex matches
2. create a new configuration phase called 'after-tests-complete' or something, and allow that to be configured to only run if a certain branch regex matches
3. move the downstream work to a separate configuration that is triggered by the successful completion of my tests, and allow that configuration to also have a branch-regex-based trigger. In this case, I'd want to use a snapshot dependency probably.
Personally, I'd like to be able to do that sort of work within a single configuration, as it simplifies the agent provisioning story etc. vs. running in multiple configurations.
Also, I can imagine making this more complex (I think that some of the linked issues delve into these details). For example, if I push changes to 'pcl-remote-tests', I might want to push that to master automatically. If master has no new changes vs. what's in pcl-remote-tests, then it'd be safe to just go ahead and perform the master-oriented deployment work.
2. Ветки создаются отдельными девелоперами по необходимости, здесь нет централизованного требования, за исключением того, что только те ветки, которые полезны всем должны находить своё отражение в публичных конфигурациях ТС. То есть личные экспериментальные ветки в ТС попадать не должны. Необходимость публичности бранчей в ТС объясняется ещё одной методологической особенностью нашего процесса - каждый член команды может взять подзадачу из любой разрабатываемой ветки любого продукта, чтобы её продолжить. Таким образом, публичность результатов, в частности тестов, является необходимостью.
3. Ветки могут жить от нескольких дней до нескольких месяцев. Как ясно из п.2 вопрос об "интересности" бранча для отд. разработчиков не стоит, поэтому ограничение видимости, на мой взгляд будет только вредить.
4. Интеграция изменений между бранчами. Не очень понятно, какое отношение имеет этот вопрос к ТС. На сегодня, интеграция (merge) осуществляется средствами Mercurial per se и визуализируется с помощью, например, Hg Workbench. Факт мержа отражается, в том числе, в комментариях к коммитам. Поэтому, для ТС эти мержи, по идее, не должны ничем отличаться от обычных коммитов, если только вы не хотите рисовать параллельное дерево с Hg (может, это и не плохо).
5. Закрытая ветка - это закрытая ветка. Моё личное мнение - соответствующая build-конфигурация должна быть удалена, возможно не сразу. Возможно, после закрытия она должна быть видима только автору брэнча. Ветки ведь закрываются не просто так, их результат сливается в другую ветку (не обязательно транк), после чего соответствующая функциональность живёт совместной жизнью с другими. Рабочие ветки вряд ли метятся тагами, если только для собственных нужд, в основном, у нас сейчас принята практика отмечания (пред)релизных билдов в QA-ветке. В конце концов вся информация остаётся доступной в репозитории, и, если кто-то захочет порыться в "анналах", всегда можно локально сапгрейдиться (во блин, слово...) на нужную версию закрытой ветки и поиграться.
6. Возможность настраивать нотификации по билдам. Да, вот нотификации, мне кажется, не должны приходить всем. Только тем, кто в них сейчас заинтересован.
7. Взаимосвязь между настройками базовой конфигурации и её потомками для других бранчей мне кажется самым мутным вопросом. На первый взгляд - безусловно настройки должны наследоваться и синхронизироваться. С другой стороны, не могу привести обратный пример. Я думаю, этот момент мне надо обговорить с коллегами, чтобы прийти к более конкретному мнению.
if the "personal" option supported a "public" mode checkbox we'd be 90% of the way there by using "/refs/heads/project-*".
http://nvie.com/posts/a-successful-git-branching-model/
In git-flow it is common for features to be "feature/some-feature-name", hotfix branches are "hotfix/X.Y.Z" and release branches are "release/X.Y"
Last week I just got annoyed with having to manually create build configurations and reminding people to delete their obsolete configurations, I just wrote an app that syncs our git development branches with a TeamCity project. I used the REST API to read details from TeamCity and had to just POST to the URLs that the dashboard uses to create and delete configurations because the REST API doesn't expose that functionality. It seems to work well enough for us at the moment, but I would be happy to move over to an inbuilt and supported feature.
http://jonorossi.com/blog/archive/2011/11/28/sync-teamcity-build-configurations-with-git-branches.aspx
I apologise to the JetBrains staff if we are going too off topic, so if you want to discuss the code please use my blog or the gist comments.
What we want to do in the scope of this issue is to pursue with the approach outlined in the issue description: allow to build different branches in the scope of a single build configuration. With this approach it would probably make sense to have a single development build for all branches and separate one for master to run more builds (like deployment) on it's base.
Dear voters, can you please look through below and comment which one of the approaches you would consider better for you?
Rough approaches comparison:
also, there are points which will probably be easier to add in single build configuration approach:
Currently I use a script to add/remove project definitions to the configuration file (using a small tool: https://gist.github.com/1401596) that sets the project name (the branch name) that the template they are based on uses to determine what branch to sync for the build. I have this run every 5 minutes and use 'git ls-remote' to determine the branches that should be present (piped into a file the tool reads).
We set up different build triggers on branches and need to be able to modify the build configuration parameters from the template for specific branches. I would also expect our users would like to be able to configure notifications only for their branches and configure their dashboard the same way.
Most probably no (at least for a start). However, you will be able to get current branch name inside the build.
Clifford mentions the same approach above.
Is this a critical obstacle for your case?
As to "need to have some branches builds customized":
If you need to change some settings for specific branches, then that means you start to manage them manually and there is no (at least obvious) way to generate those build configurations automatically.
The most common case for "Single Build Configuration" approach seems to include separate build configuration for "master" branch with specific settings and separate build configuration for "all the other feature branches" that do not need any manual tweaking. If you need a manual setting for some branches you can copy a build configuration, set branch name and do change the settings - and this copying logic is _already_ possible.
Are there cases when you need automatic builds for newly appearing branches, but still need to change settings for some of them?
Hope that makes it more clear.
It's also not safe to assume any particular branching convention, especially with Mercurial (Mercurial users may be using Bookmarks, or Repository-based Branching instead of named branches, even though named branches are the only branching model really well supported right now).
Are you storing your build configurations in TC? I'm suggesting that TC should know very little about the differences in the builds and the branches. I setup a script to add new branches based on a template. The template does:
1. sync the code (from branch, repo, bookmark, whatever)
2. set BRANCH_NAME= in the environment
3. run "build.bat" in the sync'd code to build all targets/configurations that are valid for the build agent's environment
The logic for building the branch lives in the branch, as does the code. That logic changes when the code changes require it.
How you determine what a "branch" is (a repo, bookmark, etc.) won't affect the usage pattern. What's important is the code changes that require a change to the build system are paired with the changes to the build system.
So if you have 100 targets/configurations and 100 branches off of master the only branches with different build logic are those that have added code that need changes to the build logic. Most branches will have identical build scripts (or ones with minor changes from that of the parent branch), and the TC template for all of them is the same.
This request (TW-16052) is about building in support for this usage pattern (building branches without having to manually set them up as a new build) into TC directly so we don't have to write an external script to list the branches in the VCS and update the project configuration files to reflect them. (Which is what I ended up doing anyways...)
---
There is one TeamCity project (call it 'Trunk') that always builds the 'mainline' codebase. Branches are made off of this mainline for each feature. Bob, Alice, and Eve all have feature branches. Each one of of them has a TeamCity project that has the same build configurations as the Trunk project, but the VCS root points to his/her branch.
Eve finishes some work that makes 3 of the build configurations have changed snapshot and artifact dependencies, and adds a new build configuration. She merges her branch into 'maneline' in the VCS, updates the TeamCity Trunk project with the correct artifact/snapshot dependency changes from her own TeamCity project (by hand, because there is no automatic way to do this, which is very cumbersome), and retires her branch, both in the VCS and TeamCity.
Alice and Bob both merge Trunk into their branches. However, they now have Eve's code changes, so they now need the changes that Eve made to Trunk's build configurations to use different artifact/snapshot dependencies. So they have to also make these changes manually in their own TeamCity projects, by hand, becuase there is no automatic way to do it, which is very cumbersome. They also have the option of deleting their TeamCity projects and creating new ones, but they lose the build history of possibly several weeks of work.
---
Now imagine the team is not just made up of Alice, Bob, and Eve, but instead of 75 developers, with 50 active branches, and every TeamCity project has 70+ build configurations.
Our team hits this scenario all the time and it's driving us crazy. I am now in the process of writing a tool to let us manipulate TeamCity projects through TeamCity's REST API that shipped with 7.0, but I don't feel like I should have to.
Regarding assumptions about branching, my comment was about the current assumption that Mercurial users are using named branches for branching. In Git, this pretty much a universally safe assumption, but it isn't with Mercurial – yet the remote build feature in TeamCity for Mercurial and Git assume you're willing to push to a named branch and VCS roots for Mercurial don't supply an option to support bookmarks, only named branches.
I sounds like you're using TeamCity to manage artifact dependencies between products, which I've avoided doing so far.
W.r.t. this feature request, I'm not clear if you're arguing for it (the ability to build multiple check outs from different "branches" of a product using one TC build configuration), against it, or for something more than it.
W.r.t this feature request, I'm arguing that it's not enough to do what's in the feature request and call it "support for feature branches in DVCS". From the original feature request: "The goal is to have single set of build configuration settings but allow it to build many branches". This may work for simple cases, but it's not a global solution to managing branches in DVCS.
Sometimes build configurations need to change slightly based on changes made in the branch (maybe you tweaked a dependency list for an installer? Maybe you upgraded the .NET framework requirement and need to specify it run on a certain set of agents?)
Another issue with this approach is that it's valuable to be able to see only the build history for branch X when you have a long-lived branch, which conflicts with the idea of having one project build multiple branches (though this could be solved by adding a "filter by VCS root" or "filter by branch" option in this history view.