• 
      

    Improve query optimization when using ClearCase-based repositories.

    Review Request #8936 — Created May 9, 2017 and submitted

    Information

    RBTools
    master

    Reviewers

    We optimize HTTP requests to Review Board by prioritizing repositories
    based on name. Previously this was done using strict comparison of name
    returned by Review Board to name returned by cleartool. This is no good
    when supporting users on Windows and Unix both. They are different, as
    shown here:
    
        W:\srcSHARE>cleartool describe -short vob:.
        \srcSHARE
    
    And when the VOB is mounted to /vobs in Unix:
    
        [/vobs/srcSHARE]$ cleartool describe -short vob:.
        /vobs/srcSHARE
    
    Now we compare the name of the returned repository with each part of
    the VOBs tag returned by cleartool. Subsequently, Review Board server
    administrators can set repository names for individual VOBs and can
    expect it to match for users regardless of platform. `srcSHARE` works
    in the example above.

    I posted a review to an internally-hosted Review Board instance from a
    VOB with the name srcSHARE on both Linux and Windows. The repository
    in Review Board has the name srcSHARE.

    In the debug output it is clear that the number of repositories queried
    is reduced. In our specific case because the Review Board server is
    hosted very far away, this represents a time savings of a few minutes
    for each review posted.

    Before fix (RHEL):

    (project_chunt) [server /vobs/srcSHARE/project]$ rbt post -d hello.txt@@/main/featurebranch1/2:hello.txt@@/main/featurebranch1/3
    ...
    >>> Checking for a ClearCase repository...
    >>> Running: cleartool pwv -short
    >>> Running: diff --version
    >>> Running: cleartool lsview -full -properties -cview
    >>> Running: cleartool describe -short vob:.
    >>> Running: cleartool pwv -root
    >>> repository info: Path: /vobs/srcSHARE, Base path: /vobs/srcSHARE, Supports changesets: False
    >>> Making HTTP GET request to http://company.local:8888/api/
    >>> Running: cleartool lsvob -long /vobs/srcSHARE
    >>> Repository's /vobs/srcSHARE uuid is u'0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa'
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?max-results=25&start=25&tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/5/info/
    >>> Got API Error 210 (HTTP code 500): There was an error fetching extended information for this repository.
    >>> Error data: {u'stat': u'fail', u'err': {u'msg': u'There was an error fetching extended information for thisrepository.', u'code': 210}}
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/7/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/178/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/9/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/168/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/169/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/170/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/171/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/172/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/173/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/174/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/175/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/176/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/36/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/72/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/179/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/180/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/181/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/182/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/183/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/184/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/185/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/88/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/90/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/101/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/106/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/109/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/119/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/129/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/131/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/138/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/177/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/165/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/166/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/164/info/
    >>> Matching repository uuid:0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa with path:/view/rb_srcshare/vobs/srcSHARE
    >>> repository info: Path: /view/rb_srcshare/vobs/srcSHARE, Base path: /view/rb_srcshare/vobs/srcSHARE, Supports changesets: False
    ...
    

    After fix (RHEL):

    (project_chunt) [server /vobs/srcSHARE/project]$ rbt post -d hello.txt@@/main/featurebranch1/2:hello.txt@@/main/featurebranch1/3
    ...
    >>> Checking for a ClearCase repository...
    >>> Running: cleartool pwv -short
    >>> Running: diff --version
    >>> Running: cleartool lsview -full -properties -cview
    >>> Running: cleartool describe -short vob:.
    >>> Running: cleartool pwv -root
    >>> repository info: Path: /vobs/srcSHARE, Base path: /vobs/srcSHARE, Supports changesets: False
    >>> Making HTTP GET request to http://company.local:8888/api/
    >>> Running: cleartool lsvob -long /vobs/srcSHARE
    >>> Repository's /vobs/srcSHARE uuid is u'0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa'
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?max-results=25&start=25&tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/164/info/
    >>> Matching repository uuid:0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa with path:/view/rb_srcshare/vobs/srcSHARE
    >>> repository info: Path: /view/rb_srcshare/vobs/srcSHARE, Base path: /view/rb_srcshare/vobs/srcSHARE, Supports changesets: False
    ...
    

    Before fix (Windows):

    W:\srcSHARE\project>rbt post -d hello.txt@@\main\featurebranch1\2:hello.txt@@\main\featurebranch1\3
    ...
    >>> Checking for a ClearCase repository...
    >>> Running: cleartool pwv -short
    >>> Running: diff --version
    >>> Running: cleartool lsview -full -properties -cview
    >>> Running: cleartool describe -short vob:.
    >>> Running: cleartool pwv -root
    >>> repository info: Path: W:\srcSHAR, Base path: W:\srcSHAR, Supports changesets: False
    >>> Making HTTP GET request to http://company.local:8888/api/
    >>> Running: cleartool lsvob -long \srcSHARE
    >>> Repository's \srcSHARE uuid is u'0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa'
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?max-results=25&start=25&tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/5/info/
    >>> Got API Error 210 (HTTP code 500): There was an error fetching extended information for this repository.
    >>> Error data: {u'stat': u'fail', u'err': {u'msg': u'There was an error fetching extended information for this repository.', u'code': 210}}
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/7/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/178/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/9/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/168/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/169/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/170/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/171/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/172/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/173/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/174/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/175/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/176/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/36/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/72/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/179/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/180/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/181/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/182/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/183/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/184/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/185/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/88/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/90/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/101/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/106/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/109/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/119/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/129/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/131/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/138/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/177/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/165/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/166/info/
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/164/info/
    >>> Matching repository uuid:0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa with path:/view/rb_srcshare/vobs/srcSHARE
    >>> repository info: Path: /view/rb_srcshare/vobs/srcSHARE, Base path: /view/rb_srcshare/vobs/srcSHARE, Supports changesets: False
    ...
    

    After fix (Windows):

    W:\srcSHARE\project>rbt post -d hello.txt@@\main\featurebranch1\2:hello.txt@@\main\featurebranch1\3
    ...
    >>> Checking for a ClearCase repository...
    >>> Running: cleartool pwv -short
    >>> Running: diff --version
    >>> Running: cleartool lsview -full -properties -cview
    >>> Running: cleartool describe -short vob:.
    >>> Running: cleartool pwv -root
    >>> repository info: Path: W:\srcSHAR, Base path: W:\srcSHAR, Supports changesets: False
    >>> Making HTTP GET request to http://company.local:8888/api/
    >>> Running: cleartool lsvob -long \srcSHARE
    >>> Repository's \srcSHARE uuid is u'0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa'
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/?max-results=25&start=25&tool=ClearCase
    >>> Making HTTP GET request to http://company.local:8888/api/repositories/164/info/
    >>> Matching repository uuid:0f270ab6.a93811d2.a4ff.08:00:09:d2:c6:aa with path:/view/rb_srcshare/vobs/srcSHARE
    >>> repository info: Path: /view/rb_srcshare/vobs/srcSHARE, Base path: /view/rb_srcshare/vobs/srcSHARE, Supports changesets: False
    ...
    
    Description From Last Updated

    Stye nit: blank line between these. Also, can you re-add the comment?

    brenniebrennie
    brennie
    1. This looks great! Thanks for the contribution. I've only got one minor issue:

    2. rbtools/clients/clearcase.py (Diff revision 1)
       
       
       
      Show all issues

      Stye nit: blank line between these.

      Also, can you re-add the comment?

    3. 
        
    CH
    CH
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-0.7.x (6f7f5a8)
    chipx86
    1. Ship It!
    2.