^cR{ddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZddlmZddlmZd Zd Zd)d Zd ZdZd*dZdZdZ d+dZ!dZ"dZ#dZ$GddZ%dZ&dZ'dZ(dZ)dZ*dZ+ej,dZ-dZ.dZ/d Z0d!Z1d"Z2d#Z3ej,d$Z4d%Z5d&Z6d'Z7d(Z8dS),N)_)open) repository) bookmarks cacheutilerror narrowspecphasespycompat requirementsscmutilstoreutil)nodemap) stringutilclt|}|tjz}|||S)a determine the final set of requirement for a new stream clone this method combine the "default" requirements that a new repository would use with the constaint we get from the stream clone content. We keep local configuration choice when possible. )setrequirementsmodSTREAM_FIXED_REQUIREMENTSupdate)default_requirementsstreamed_requirementsr s 7/usr/lib/python3/dist-packages/mercurial/streamclone.pynew_stream_clone_requirementsr%s:+,,LO==L-... c.|jtjz}|S)zthe set of requirement the new clone will have to support This is used for advertising the stream options and to generate the actual stream content.)r rr)reporequiredformatss rrr2s OEE rFcp|j}|j}d}|jrd|jdgvrd}|r|sdS|r|sdSt |rdS|jrdS|j}||d}|sdSt}|dr | tj n|d}|s.|jj td dSt|d }||jz }|r}|jj td d t'|z|jj td dS|}d|fS)aWhether it is possible to perform a streaming clone as part of pull. ``bundle2`` will cause the function to consider stream clone through bundle2 and only through bundle2. Returns a tuple of (supported, requirements). ``supported`` is True if streaming clone is supported and False otherwise. ``requirements`` is a set of repo requirements from the remote, or ``None`` if stream clone isn't supported. Fsv2sstreamT)FNNsstream-preferreds streamreqss=warning: stream clone requested but server has them disabled ,sGwarning: stream clone requested but client is missing requirements: %s , sQ(see https://www.mercurial-scm.org/wiki/MissingRequirement for more information) )rremote canusebundle2remotebundle2capsgetlenheadsstreamclonerequestedcapableraddrREVLOGV1_REQUIREMENTuiwarnrsplit supportedjoinsorted) pullopbundle2rr#bundle2supportedstreamrequestedr streamreqs missingreqss rcanperformstreamcloner9=s ;D ]F $ F,00B?? ? ?#  { ){ 4yy{|{1O ..)<== {55L ~~i  "=>>>>^^M22   KN  "    ;))$//00  4>1   KN  ***VK0011 2    KN  /    ;!  rcpddlm}t|\}}|sdS|j}|j}d}|drT|5}|di}dddn #1swxYwY|j td|5}|di}dddn #1swxYwY| } t| } n0#t$r#tjtd| wxYw| dkr!tjtd| d kr!tjtd | d kr!tjtd | } t%t| d d\} } n7#tt(f$r#tjtd| wxYw|5t-||| | t/|j||_||j |j|j|j_t;j|t?j ||r|j!"|||#ddddS#1swxYwYdS)zPossibly perform a legacy stream clone operation. Legacy stream clones are performed as part of pull but before all other operations. A legacy stream clone will not be performed if a bundle2 stream clone is supported. r localrepoNs branchmapsstreaming all changes s stream_out'unexpected response from remote server:soperation forbidden by servers$locking the remote repository failedrs%the server sent an unknown error code )$r<r9rr#r*commandexecutor callcommandresultr-statusrreadlineint ValueErrorr ResponseErrorAbortmapr/ TypeErrorlock consumev1rr resolvestorevfsoptionsfeaturessvfsoptionsrwritereporequirementsrpost_stream_cleanup _branchcachesreplace invalidate) r3r<r0r rr# rbranchmapefplresp filecount bytecounts rmaybeperformlegacystreamcloner^s3F;;I|  ;D ]FJ ~~l##B  # # % % B|R88??AAJ B B B B B B B B B B B B B B B GNN1/00111    ! !7Q ]]=" - - 4 4 6 6777777777777777 A 1vv    ! 8 9 91     qyyk!<==>>> k!CDDEEE k!DEEFFF A "3a(8(899 99  "   ! 8 9 91    $Iy1119     &<< GT&     %d+++#D)))  9   & &tZ 8 8 8 sO)B  BB)DD  D &D66-E#0,H4I(B6L++L/2L/ctj|jvrdS|jdddsdSt j|}|r|jddSdS)z5Whether streaming clones are allowed from the server.Fsservers uncompressedT) untrustedsuncompressedallowsecret)rREPO_FEATURE_STREAM_CLONErOr- configboolr hassecret)rsecrets rallowservergenerationrest+4=@@u 7  iD  I Iu d # #F Iw!!)-GHHH 4rc6|j|SN)rwalk)rmatchers r_walkstreamfilesrjs :??7 # ##rcgd}5jdtD]$\}}}|r||f||z }%t dddn #1swxYwYt jdt|fzjjj fd}t||fS)aEmit content for version 1 of a streaming clone. This returns a 3-tuple of (file count, byte size, data iterator). The data iterator consists of N entries for each file being transferred. Each file entry starts as a line with the file name and integer size delimited by a null byte. The raw file data follows. Following the raw file data is the next file entry, or EOF. When used on the wire protocol, an additional line indicating protocol success will be prepended to the stream. This function is not responsible for adding it. This function will obtain a repository lock to ensure a consistent view of the store is captured. It therefore may raise LockError. r scanning Ns%d files, %d bytes to transfer c3VKD]\}}rjd||fzdtj||fzV|dd5}|dkr||Vnt j||D]}|Vdddn #1swxYwYdS)Nssending %s (%d bytes) s%s%d rbF) auditpathlimit)r-debugr encodedirreadr filechunkiter)namesizerYchunk debugflagentriesrrPs remitrevlogdataz"generatev1..emitrevlogdata!s#! $ $JD$ I 8D$<GHHH!6!6 == = = =dEU333 $r5==''$--''''!%!3Bd!C!C!C$$#  $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $sQQ>Hrrbundlebytestotalunitrstep) rr-rDrstructpackr' makeprogressr incrementcomplete)r\r]rprogressry compressionrrequiress rgenzgeneratebundlev1..genksS #-d#3#3 9b  0 1 1Y 4J J   k&)Y77777k%X!233333e####7'' iLL ( (     E   CJJ  / / /KKKKr)rGrr1r2)rrr rrs`` @rgeneratebundlev1rOs},eJKKK(..Lyy --..H6  rc X|5|jtd|t j|fz|jtd|td}|dt j}| d5|j |j|5t|D]K}| } |dd\}} t| } n7#t t"f$r#t%jtd |wxYw|jjr1|jd |t j| fzt-j|} | | d d 5} t j|| D]:} |t5| | | ; dddn #1swxYwYM dddn #1swxYwY|d dddn #1swxYwYt j|z } | dkrd} ||jtdt j|| t j|| z fzddddS#1swxYwYdS)a Apply the contents from version 1 of a streaming clone file handle. This takes the output from "stream_out" and applies it to the specified repository. Like "stream_out," the status line added by the wire protocol is not handled by this function. !%d files to transfer, %s of data clonerrr) expectedcountrrr=sadding %s (%s) wT)backgroundcloserqrNclearfilecacheMbP?(transferred %s in %.1f seconds (%s/sec) )rLr-rDrrr]rrtimer transactionrPbackgroundclosingrangerEr/rFrGrKr rHrzrsr decodedirrvrr'writerVr)rrYr\r]rstartirZrwrxpathofpryelapseds rrMrMs ? ?   3 4 4$.334 5   7'' hKKyq{{(       h ' ' 1 1,,TWI,NN - -y))--A A%&WWUA%6%6 d"4yy& 2#1HII1w( /49M9M2NN!?400D4tDD-%)%7$%G%G%G--E$..CJJ.???IIe,,,,----------------!- - - - - - - - - - - - - - - -0 OO4O 0 0 03 1 1 1 1 1 1 1 1 1 1 1 1 1 1 16*,,& a<<G  : ; ;y))y7233    q? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? sB3L"J*'I(D;:I;4E/ /A,IAH: . I:H>>IH>I JI JI J5 LJ LJ B LL#&L#c<|d}|dkr$tjtd|zt jd|d\}}t jd|dd}||}|ds!tjtd t|d d }|||fS) Nr>rs<only uncompressed stream clone bundles are supported; got %srrrrs@malformed stream clone bundle: requirements not properly encodedr!) rur rIrrunpackendswithrrstripr/)rYrr\r] requireslenrr s rreadbundle1headerrs''!**Kek %       "===Iy-rwwqzz2215Kww{##H   U # # k 5     xu--33D99::L i --rcxt|r!tjtdt |\}}}||jz }|rDtjtddt|zt||||tj |dS)zApply the content from a stream clone bundle version 1. We assume the 4 byte header has been read and validated and the file handle is at the 2 byte compression identifier. s2cannot apply stream clone bundle on non-empty repo4unable to apply stream clone: unsupported format: %sr"N) r'r rIrrr0r1r2rMrrS)rrYr\r]r r8s r applybundlev1rs  4yy k C D D   *;2)>)>&Iy,/K k E F Fjj ,,-- .   dB 9--- %%%%%rceZdZdZdZdZdS)streamcloneapplierzClass to manage applying streaming clone bundles. We need to wrap ``applybundlev1()`` in a dedicated type to enable bundle readers to perform bundle type-specific functionality. c||_dSrg)_fh)selffhs r__init__zstreamcloneapplier.__init__s rc,t||jSrg)rr)rrs rapplyzstreamcloneapplier.applysT48,,,rN)__name__ __module__ __qualname____doc__rrrrrrs< -----rrscc\g}|s|d|S)z!list snapshot file from the stores phaseroots) publishingr})rfnamess r_walkstreamfullstorefilesrs0 F ??  % m$$$ Mrcz|\}}}}|tkr|S|||||||fS)z actually copy the snapshot files) _filefullr1)entrycopyvfsmapsrcrwftypedatas r _filterfullr&sK"Cud  udd6#;#3#3D#9#9:: ;;rc#Kgtjd fd}|VD]}tj|tjdS#D]}tj|tjwxYw)z(return a function to temporary copy files hg-clone-)prefixctjtj|\}}tj||tj||d|S)N)rdirT)hardlink) r mkstemposrbasenamecloser}r copyfiles)rfddstdst_dirfiless rrzmaketempcopies..copy6sp&w'',,'GB HRLLL LL    N3d 3 3 3 3JrN)r mkdtempr tryunlinktryrmdir)rtmprrs @@rmaketempcopiesr.s El333G         C N3     g  C N3     gs A/Bctt|jt|ji}|j|vsJ|S)z$make a (src -> vfs) map for the repo) _srcstorerP _srccachecachevfsvfsvalues)rrs r_makemaprFs; 494=F 86==?? * * * * Mrc #Kt||jvrtjd|jtd|td}|dt5|5fd|D}dVd}|D]q\}}}}|} |Vtj t|V|tkr| |} |} n4|tkr)t|d} tj| j} d} tj | V|V| d kr| | f} ntj| | } | D]?}| t|z } |t|z }|||V@| | kr'tjtd | || fz | Z#| wxYw dddn #1swxYwYddddS#1swxYwYdS) zactually emit the stream bundle9repo.vfs must not be added to vfsmap for security reasonsrrrrc2g|]}t|Sr)r).0rXrrs r z_emit2..fs%AAAA;q$//AAArNrnrprqs=clone could only read %d bytes from %s, but expected %d bytes)rrrr ProgrammingErrorr-rrrrr uvarintencoder' _fileappendrrfstatst_sizerurvrIr)rr{ totalfilesizertotalbytecountrrwrrrrYrxr]chunksryrrs @@r_emit2rSs" d^^F  x6==??""$ H   w## ) M( $H OOA   (T8((AAAAAAAA &-" "  "Cud+CIII$SYY// / / / ##SYY)##$&&z"~~-I (..... 5== ggdmm-FF!/$???F#  EU+I"c%jj0NOON333KKKK$$ +1%dD1 2%  E"  (((((((((((((((((((((((((((((((((sPI BH8?B>""t  "B11  NNItR6 7 7 7 T !M)$//?? 9  D ! ! ? TY__T22: :M NNItY= > > >BTY--k::B55==  ; 4@AAA%d++?? =   % % ? T]0066> >M NNItY= > > > M !!rcb|5|jdt||||\}}t |||}t |}|Jt |dddn #1swxYwYt|t|||fS)aEmit content for version 2 of a streaming clone. the data stream consists the following entries: 1) A char representing the file destination (eg: store or cache) 2) A varint containing the length of the filename 3) A varint containing the length of file data 4) N bytes containing the filename (the internal, store-agnostic form) 5) N bytes containing the file data Returns a 3-tuple of (file count, file size, data iterator). rlrrrN) rLr-rsrrnextr~rr')rrrrr{rrfirsts r generatev2rs &&  m$$$!) / " " " g}55V }}}%%%&&&&&&&&&&&&&&&D!!! w<< ..sA$BB  B c'K|d}|dd}|5|r&t|5dVdddn #1swxYwYndVddddS#1swxYwYdS)Nrr)nested)ctxsthisrests rrrs 7D 8D                     EEE s, A4 A8 A8 AAAc 5jtd|t j|fzt j}jtd|td}|dt}j | vrtj dd5fd| D}t|5t!|D]}t j|d} || } t j|} t j|} t j|| } jjr2jd | | t j| fz| | d 5}t j|| D]:}|t/| ||; d d d n #1swxYwY d d d n #1swxYwYdd d d n #1swxYwYt j|z }|dkrd}jtdt j|j|t j|j|z fz|d d d d S#1swxYwYd S)zApply the contents from a version 2 streaming clone. Data is read from an object that only needs to provide a ``read(size)`` method. rrrrrrc3LK|]}|jVdSrg)rr-)rrrs r zconsumev2.. s3NNsC))$'22NNNNNNrrsadding [%s] %s (%s) rrqrNTrrr)rLr-rDrrr]rrrrrrr rrrr readexactlyuvarintdecodestreamrzrsrvrr'rrVposr)rrYr\filesizerrrrrrrnamelendatalenrwrryrs` r consumev2r)s, <<  3 4 4$.223 4    7'' hKKxakk(   $ 8v}} & &(L   h ' ' 1 1NNNNfmmooNNND - -y))--A*2q11C +C"6r::G"6r::G+B88Dw( 4"D$.*A*ABC T4-C%)%7'%J%J%J--E$..CJJ.???IIe,,,,----------------- - - - - - - - - - - - - - - -, OO4O 0 0 01 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14*,,& a<<G  : ; ;x|,,x|g566     y<<<<<<<<<<<<<<<<<<sC1L6*J1B7I!(AI ; I!I  I!I I! J!I% %J(I% )J L6J L6J BL66L:=L:cddlm}fd|D}|rDtjt ddt |zt|||tj |_ | j j j j _tjt#jdS)Nrr;c&g|] }|jv |Sr)r0)rrrs rrz!applybundlev2..5s%FFFat~.E.E1.E.E.Errr")r@r<r rIrr1r2r)rr rNr-rOrPrQrrRrrS)rrYr\r&r r<r8s` r applybundlev2r-2sFFFFlFFFK k E F Fjj ,,-- .   dB 8,,,5 D"88 "DMDI !$''' %%%%%rc dg fd}|D]\}}}||}||} ||} | |} tj| } tj| st j| | |t j| | d|d ۉ dS)NTc:dd<td_dS)NFrscopying)rtopic)rrsr copy_usedz_copy_files..copy_usedLs :rrF)rno_hardlink_cbcheck_fs_hardlink) r1rrdirnamer rmakedirs register_filecopyfiler) src_vfs_map dst_vfs_mapr{rr1krrxsrc_vfsdst_vfssrc_pathdst_pathr4rs ` @r _copy_filesr?IsvH''''''! 4a.a.<<%%<<%%'//(++w~~g&& # M' " " "d###  a[$#       A;rc d|jD}d|jD}||ksJ|5|5tj|}|d}d}t j|rd}t|ddd\}}t|} t|} |j td t||ztd  } d |D} t| | | | } |r>tj|}|d}tj||dddn #1swxYwY| | rd }nd}|j |t||zz|d5}|j|dddn #1swxYwY|jdfg}|||D]\}} ||#t4$rx}|jt6jkrYtd}||}t;j|}|||fz}|j |Yd}~d}~wwxYw ddddS#1swxYwYdS)zYcopy all content from one local repository to another This is useful for local clonec.h|]}|tjv|SrrWORKING_DIR_REQUIREMENTSrr,s r zlocal_copy..ks0 O< < < < < .ps0 O< < < < < .s1II)=D%at_IIIIIIrslinked %d files scopied %d files s localclonesundo.backupfilesserror removing %s: %s ) r rLr bookmarksvfsr1rrr rrr-rrr'r?rr7rrsrrrrPextend undofilesunlinkOSErrorerrnoENOENTr forcebytestrr.)src_repo dest_reposrc_store_requirementsdest_store_requirements src_book_vfs srcbookmarksbm_countr{rr8 dest_vfs_maprrr dst_book_vfs dstbookmarksmsgtr undo_filesundovfsundofilerXre_msgs r local_copyr`gs& ' "%< < < < <   >+>+ ]]__& :& : %1(;;L',,\::LHw~~l++ %-"& &&& "G] #8,,K#I..L{// mm'llX-x[[0HJIIIIE"; eXNNH :(5i@@ +00>>  lL999M& :& :& :& :& :& :& :& :& :& :& :& :& :& :& :N   '&CC&C #W!89:::  " "= 1 1 &R O ! !" % % % & & & & & & & & & & & & & & &!~':;< )--//000!+ + + GX +x(((( + + +7el**677C"<<11D&3A66ED%=(CL%%c***  + +k>+>+>+>+>+>+>+>+>+>+>+>+>+>+>+>+>+>+sLD&F: LF L F A L.H LH LH ;LI/.L/ K19A.K,'L,K11LL L)Frg)r)9 contextlibrMrri18nrr r interfacesrr@rrr r r r rrrr revlogutilsrutilsrrrr9r^rerjrrrrMrrrrrrrrrcontextmanagerrrrr~rrrrr)r-r?r`rrrrgs2 """"""                           ZZZZzHHHV$$$$$575757p87777tH H H V...8&&&. - - - - - - - -    <<< .   999x666666&"&"&"R///B BBBJ&&&.