History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: CIB-2870
Type: Improvement Improvement
Status: Open Open
Priority: 3 3
Assignee: Unassigned
Reporter: Job Mulder
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Pulse

Check dependency before scheduling builds from SVN trigger

Created: 06/Feb/12 09:27 AM   Updated: 07/Feb/12 02:52 AM
Component/s: Scm, Scheduling
Affects Version/s: 2.4.15
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown

Severity: Major


 Description  « Hide
It would be nice if Pulse would check the dependencies of projects before scheduling individual builds.

Suppose I have project A, B and C, B depends on A and C depends on B.
Suppose also that I have made a change in A that cascades through B to C (say an interface change of a commonly used object). When I commit this change, the logical build order would be A->B->C.
When I commit these changes (1 commit, they're in the same repository), Pulse does not check these dependencies and will in fact see that these 3 projects have 'unbuilt' changes so it will schedule these:
A: svn trigger
B: svn trigger
B: dependent on A
C: svn trigger
C: dependent on A
C: dependent on B

As a result, project C will be built 3 times (2 of which will probably fail because they expect the interface change I made in A), B will be built twice (also probably failing the first time). Whether or not B and C will fial to build the first times depends on the (seemingly random) order in which they are scheduled.

I would like to propose that when Pulse detects a change in multiple projects and they have the same revision, that the dependency tree is walked upstream and only the highest projects (which the others depend on) get triggered. Note that it could be that there is no single common ancestor. (ie project C depends on B, but B does not depend on A, in thsi case C will still get built twice because both A and B will be triggered, but that's still an improvement)

 All   Comments   Work Log   Change History      Sort Order:
Daniel Ostermeier - 06/Feb/12 06:05 PM
Hi Job,

Some (if not all) of the above should already be in place.

From the perspective of the SCM trigger, the order in which the trigger events are generated will depend on both what projects are to be triggered and what there dependency ordering is. So in the case of a single commit across multiple projects, you should see the triggers in the expected dependency order.

Now the order in which the triggers are generated is not the final decider in the order in which jobs are run, but is the first step.

When the triggers reach the job scheduler, it will do its best to order them according to dependencies, merging requests that it considers from the same source (commit / or manual trigger) if it believes the outcome will be the same.

So, given that I think what you describe should already work, the next question is how have you configured your projects (a,b,c)
a) What are the dependency relationships you have defined and how are they configured.
b) Are your changes made as a single commit (sound like it)

Cheers,
-Daniel


Job Mulder - 07/Feb/12 02:52 AM
Hi Jason,

a1) I have the dependencies set up through the project deoendencies page for each project, set to corresponding stages, usually transitive.
a2) A dependent build trigger is configured for all, with all 3 checkmarks disabled, no other properties or trigger conditions are in place for the dependent build trigger
a3) An SVN trigger is configured for each project, no additional properties have been defined, the trigger condition is set to 'outstanding changes'

b) yes, often they are. In some cases it may be that we commit A, followed shortly by B then by C

I have seen this many times, so either I have a fault in my configuration (not the first time) or there is a bug in the implementation :)