fbdZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z gdZdZdZdZdZd Zd Zej-ej.j0ej.j0j3Dcic]}||j4c}Zd Zd Zej<d j>Z ej<djBZ"ej<dZ#ej<dZ$hdZ%d>dZ&de'de'fdZ(GddejRjTZ+dZ,e+fdZ-e+fdZ.Gddej^Z0dZ1GddZ2 ddl3Z3Gd d!e2Z4ejkd!Gd"d#e7Z8Gd$d%e8Z9Gd&d'e8Z:Gd(d)e8Z;Gd*d+e8Z<Gd,d-e8Z=Gd.d/e8Z>Gd0d1e8Z?Gd2d3e?Z@Gd4d5e?ZAGd6d7e?ZBGd8d9e8ZCGd:d;e8ZDGd<d=eEeCZFe8ZGycc}w#e6$rYwxYw)?a HTTP/1.1 client library HTTPConnection goes through a number of "states", which define when a client may legally make another request or fetch the response for a particular request. This diagram details these state transitions: (null) | | HTTPConnection() v Idle | | putrequest() v Request-started | | ( putheader() )* endheaders() v Request-sent |\_____________________________ | | getresponse() raises | response = getresponse() | ConnectionError v v Unread-response Idle [Response-headers-read] |\____________________ | | | response.read() | putrequest() v v Idle Req-started-unread-response ______/| / | response.read() | | ( putheader() )* endheaders() v v Request-started Req-sent-unread-response | | response.read() v Request-sent This diagram presents the following rules: -- a second request may not be started until {response-headers-read} -- a response [object] cannot be retrieved until {request-sent} -- there is no differentiation between an unread response body and a partially read response body Note: this enforcement is applied by the HTTPConnection class. The HTTPResponse class does not enforce this state machine, which implies sophisticated clients may accelerate the request/response pipeline. Caution should be taken, though: accelerating the states beyond the above pattern may imply knowledge of the server's connection-close behavior for certain requests. For example, it is impossible to tell whether the server will close the connection UNTIL the response headers have been read; this means that further requests cannot be placed into the pipeline until it is known that the server will NOT be closing the connection. Logical State __state __response ------------- ------- ---------- Idle _CS_IDLE None Request-started _CS_REQ_STARTED None Request-sent _CS_REQ_SENT None Unread-response _CS_IDLE Req-started-unread-response _CS_REQ_STARTED Req-sent-unread-response _CS_REQ_SENT N)urlsplit) HTTPResponseHTTPConnection HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady BadStatusLine LineTooLongRemoteDisconnectederror responsesPiUNKNOWNIdlezRequest-startedz Request-sentids[^:\s][^:\r\n]*s\n(?![ \t])|\r(?![ \t\n])z[- ]z[-]>PUTPOSTPATCHc  |jdS#t$rl}t|j|j|j|j |j d||j|j dd|ddd}~wwxYw)z bytes "should" be present for reading. If the bytes are truly not available (due to EOF), then the IncompleteRead exception can be used to detect the problem. )rRrr=r )rArr&s r)rzHTTPResponse._safe_readys;ww||C  t9s? s3t9}5 5 r6ct|}|jj|}||krtt |d|||z |S)z2Same as _safe_read, but for reading into a buffer.N)r=rRrr r)rArrrBs r)rzHTTPResponse._safe_readintosF!f GG  Q  s7 q!us1u5 5r6c|j|jdk(ry|jr|j|S|j |dks||jkDr |j}|jj |}|s|r|j |S|j:|xjt|zc_|js|j |S)zvRead with at most one underlying system call. If at least one byte is buffered, return that instead. rr6r)rRrhrn_read1_chunkedrpread1r~r=)rArBresults r)rzHTTPResponse.read1s 77?dllf4 <<&&q) ) ;; "AT[[ Aq!!      [[ $ KK3v; &K;;  " r6c|j|jdk(ry|jr|j|S|jj |S)Nrr6)rRrhrn _peek_chunkedpeek)rArBs r)rzHTTPResponse.peeksE 77?dllf4 <<%%a( (ww||Ar6c|j|jdk(ry|jrt||S|j |dks||j kDr |j }|jj |}|s|r|j |S|j :|xj t|zc_|j s|j |S)Nrr6r)rRrhrnrrOrpr~r=)rAlimitrrs r)rOzHTTPResponse.readlines 77?dllf4 <<7#E* * ;; " UT[[5HKKE!!%(%      [[ $ KK3v; &K;;  " r6c|j}||dk(ryd|cxkr|ksn|}|jj|}|xjt |zc_|s t d|Sr)rrRrror=r )rArBrors r)rzHTTPResponse._read1_chunkedsj))+  aQ$*$Aww}}Q 3t9$ % % r6c |j}|y|jj|d|S#t$rYywxYw)Nr6)rr rRr)rArBros r)rzHTTPResponse._peek_chunkedsP --/J  ww||J' 44   s 3 ??c6|jjSr)rRfilenors r)rzHTTPResponse.filenosww~~r6c|j t|jj|xs|}t|ts t |ds|Sdj |S)axReturns the value of the header matching *name*. If there are multiple matching headers, the values are combined into a single string separated by commas and spaces. If no matching header is found, returns *default* or None if the *default* is not specified. If the headers are unknown, raises http.client.ResponseNotReady. __iter__z, )rSrget_all isinstanceryhasattrrX)rAr'defaultrSs r) getheaderzHTTPResponse.getheadersW << "$ $,,&&t,7 gs #77J+GN99W% %r6ct|j tt|jjS)z&Return list of (header, value) tuples.)rSrlistrrs r) getheaderszHTTPResponse.getheaderss- << "$ $DLL&&())r6c|SrrJrs r)rzHTTPResponse.__iter__s r6c|jS)ajReturns an instance of the class mimetools.Message containing meta-information associated with the URL. When the method is HTTP, these headers are those returned by the server at the head of the retrieved HTML page (including Content-Length and Content-Type). When the method is FTP, a Content-Length header will be present if (as is now usual) the server passed back a file length in response to the FTP retrieval request. A Content-Type header will be present if the MIME type can be guessed. When the method is local-file, returned headers will include a Date representing the file's last-modified time, a Content-Length giving file size, and a Content-Type containing a guess at the file's type. See also the description of the mimetools module. )rSrs r)infozHTTPResponse.infos*||r6c|jS)aZReturn the real URL of the page. In some cases, the HTTP server redirects a client to another URL. The urlopen() function handles this transparently, but in some cases the caller needs to know which URL the client was redirected to. The geturl() method can be used to get at this redirected URL. )rtrs r)geturlzHTTPResponse.geturls xxr6c|jS)zuReturn the HTTP status code that was sent with the response, or None if the URL is not an HTTP URL. )rlrs r)getcodezHTTPResponse.getcodes {{r6)rNNr))"rGrHrIrurrrr~rrrrrrrrrrrrrrrrOrrrrrrrrr __classcell__rs@r)rrs#<'BF#P: #  &P D 0;$:0 &"  5 &(* . r6rctj}|dk(r|jdg|jd|_|S)Nrzhttp/1.1T)ssl_create_default_https_contextset_alpn_protocolspost_handshake_auth) http_versioncontexts r)_create_https_contextr$sC//1Gr""J<0"".&*# Nr6c eZdZdZdZeZeZdZ dZ e dZ e dZ dejddfd Zd"d Zd Zd Zd ZdZdZdZdZdZdZdZd#dZ d$dZdZdZdZ dZ!dZ"d%dddZ#difdddZ$d Z%d!Z&y)&rrzHTTP/1.1r;rc6t|tjS)zFTest whether a file-like object is a text or a binary stream. )rio TextIOBase)streams r) _is_textIOzHTTPConnection._is_textIO;s&"--00r6c||jtvryyt|dry t|}|jS#t $rYnwxYwt |tr t|Sy)aGet the content-length based on the body. If the body is None, we set Content-Length: 0 for methods that expect a body (RFC 7230, Section 3.3.2). We also set the Content-Length for any method if the body is a str or bytes-like object and not a file. Nrr) upper_METHODS_EXPECTING_BODYrrnbytes TypeErrorrryr=)bodyrsmvs r)_get_content_lengthz"HTTPConnection._get_content_lengthAsq <||~!88 4  D!B99     dC t9 sA A  A N cR||_||_||_d|_g|_d|_t |_d|_d|_ d|_ i|_ d|_ |j||\|_|_|j!|jt"j$|_yr)timeoutsource_address blocksizerr_buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterh _tunnel_host _tunnel_port_tunnel_headers_raw_proxy_headers _get_hostporthostport_validate_hostsocketcreate_connection_create_connection)rArrrrrs r)ruzHTTPConnection.__init__as ,"      !"&!%!3!3D$!?DI DII&#)":":r6c|jr td|j||\|_|_|r|j |_n|j jtd|j DsI|jjdjd}d||jfz|j d<yy)aSet up host and port for HTTP CONNECT tunnelling. In a connection that uses HTTP CONNECT tunnelling, the host passed to the constructor is used as a proxy server that relays all communication to the endpoint passed to `set_tunnel`. This done by sending an HTTP CONNECT request to the proxy server when the connection is established. This method must be called before the HTTP connection has been established. The headers argument should be a mapping of extra HTTP headers to send with the CONNECT request. As HTTP/1.1 is used for HTTP CONNECT tunnelling request, as per the RFC (https://tools.ietf.org/html/rfc7231#section-4.3.6), a HTTP Host: header must be provided, matching the authority-form of the request target provided as the destination for the CONNECT request. If a HTTP Host: header is not provided via the headers argument, one is generated and transmitted automatically. z.Can't set up tunnel for established connectionc3BK|]}|jdk(yw)rNr<).0headers r) z,HTTPConnection.set_tunnel..sO6<<>V+Osidnaasciiz%s:%dHostN) rr RuntimeErrorrr r copyr clearanyrrY)rArrrS encoded_hosts r) set_tunnelzHTTPConnection.set_tunnelxs, 99OP P/3/A/A$/M,4, #*<<>D  & & (O$:N:NOO,,33F;BB7KL+2d//61,1D  (Pr6cH|K|jd}|jd}||kDr t||dzd}|d|}n |j}|r|ddk(r |ddk(r|dd}||fS#t$r/||dzddk(r |j}nt d||dzdzYdwxYw) Nr:]r;rxznonnumeric port: '%s'r[r)rfindrr} default_portr )rArrrjs r)rzHTTPConnection._get_hostports < 3A 3A1uOtAaCDz?D BQx(( DGsNtBx3":Dd|"OAaCDzR'#00()@4!:)MNNOsA))5B! B!c||_yr)rg)rAlevels r)set_debuglevelzHTTPConnection.set_debuglevels r6c.d|vr|ddk7rd|zdzS|S)N:r[r/r0rJ)rAips r) _wrap_ipv6zHTTPConnection._wrap_ipv6s( 2:"Q%7*"9t# # r6cd|j|jjd|j|jjdfz}|g}|j j D]+\}}|j|d|djd-|jd|jdj|~|j|j|j } |j\}}}t|j|_|j"d kDr+|j D]}t%d |j'|t(j*j,k7r/|j/t1d |d |j3 |j/y#|j/wxYw)NsCONNECT %s:%d %s rrz:  rrMr6rsrrzTunnel connection failed:  )r1r rr  _http_vsn_strr rr@sendrXresponse_classrrrhrrTrRr rgrzrYhttp HTTPStatusOKrOSErrorr) rAconnectrSrrresponserkrmessages r)_tunnelzHTTPConnection._tunnels) OOD--44V< =       % %g .-00)!11779 GMFE NNfXRwd3::9E F Gw #((7#$ &&tyy&F '/'<'<'> $WdG&3HKK&@D #""556F)V]]_56t)))  :4&'--/ARSTT* NN HNN s 9B4F??GcH|jt|jSdS)z Returns a dictionary with the headers of the response received from the proxy server to the CONNECT request sent to set the tunnel. If the CONNECT request was not sent, the method returns None. N)r r`rs r)get_proxy_response_headersz)HTTPConnection.get_proxy_response_headerss/&&2  7 7 8  r6ctjd||j|j|j |j|jf|j |j |_ |jjtjtjd|jr|j!yy#t$r(}|jtjk7rYd}~Jd}~wwxYw)z3Connect to the host and port specified in __init__.zhttp.client.connectr;N)sysauditrrrrrrr setsockoptr IPPROTO_TCP TCP_NODELAYr<errno ENOPROTOOPTr r@rAes r)r=zHTTPConnection.connects 'tyy$))D++ YYtyy !4<<1D1DF   II !3!3V5G5G K    LLN   ww%+++, s09C C8C33C8ct|_ |j}|rd|_|j|j}|rd|_|jyy#|j}|rd|_|jwwxYw)z(Close the connection to the HTTP server.N)rr rrrr)rArrr>s r)rzHTTPConnection.closesr  !99D   H"& H"& s %A(Bc|j'|jr|jn t|jdkDrt dt |t|dr|jdkDr t d|j|}|r|jdkDr t d|j|jx}rc|r|jd}tjd|||jj||j|jx}rcytjd|| |jj|y#t$r`t!|t"j$j&r$|D]}|jj|Yytd t)|zwxYw) zSend `data' to the server. ``data`` can be a string object, a bytes object, an array object, a file-like object that supports a .read() method, or an iterable object. Nrzsend:rzsending a readableencoding file using iso-8859-1rVzhttp.client.sendz9data should be a bytes-like object or an iterable, got %r)rr auto_openr=rrgrzr{rrrrrrDrEsendallrr collectionsabcIterabletype)rAr&r datablockds r)r7zHTTPConnection.sends 99 ~~ "n$ ??Q  '4: & 4 "*+__T*F$//A-67#yy88)8 ) 0 0 >I ,dI> !!), $yy88)8  $dD1 G II  d # G$  8 89)AII%%a() !9;?:!FGG  GsE AG 1G c:|jj|y)zuAdd a line of output to the current request buffer. Assumes that the line does *not* end with \r\n. N)rr@)rArs r)_outputzHTTPConnection._output(s Ar6c#@K|jdkDr td|j|}|r|jdkDr td|j|jx}r6|r|j d}||j|jx}r5yyw)Nrzreading a readablerOrV)rgrzrrrr)rArrrVs r)_read_readablezHTTPConnection._read_readable/s ??Q  & '* doo) 2 3#==88i8%,,\: O$==88i8s BBBFc|jjddj|j}|jdd=|j||t |dr|j |}n t ||f}|D]e}|s|jdkDr td |r3|jdk(r$t|d d jd |zdz}|j|g|r"|jdk(r|jd yyyy#t$r4 t|}n$#t$rtdt|zwxYwYwxYw) zSend the currently buffered request and clear the buffer. Appends an extra \r\n to the buffer. A message_body may be specified, to be appended to the request. )r6r6rMNrzAmessage_body should be a bytes-like object or an iterable, got %rrzZero length chunk ignoredrXr3rs0 )rextendrXr7rr[rriterrUrgrz _http_vsnr=r)rA message_bodyencode_chunkedrichunkschunks r) _send_outputzHTTPConnection._send_output:se J'll4<<( LLO #  #|V,,,\:- |,+_F !*9:!dnn&:"5z!nD188AEI!"E %  !$..B"6 ,'#7~O $!>>!%l!3$>')H*.|*<)=>>>>s*8 D E D,+E,!E  EEcz|jr!|jjrd|_|jtk(r t|_nt |j|j |||_|xsd}|j||d|d|j}|j|j||jdk(r&|s d}|jdrt|\}}}}}|r. |jd}|j#d t%|n|j&r|j&} |j(} n|j*} |j,} | jd} |j/| } d | vr t%| } | |j0k(r|j#d | n(| j3d} |j#d | d | |s|j#d d yyy#t $r|jd}Y wxYw#t $r| jd} YwxYw) a`Send a request to the server. `method' specifies an HTTP request method, e.g. 'GET'. `url' specifies the object being requested, e.g. '/index.html'. `skip_host' if True does not add automatically a 'Host:' header `skip_accept_encoding' if True does not add automatically an 'Accept-Encoding:' header N/r5rrxr9rrrr:zAccept-Encodingidentity)rrr r_CS_REQ_STARTEDr_validate_methodrh_validate_pathr6rY_encode_requestr`r2rrr putheaderr5r r rrr1r(rY) rArsrt skip_hostskip_accept_encodingrequestnetlocnil netloc_encrrhost_encs r) putrequestzHTTPConnection.putrequestps ??t779"DO* <<8 #*DL#DLL1 1 f% jS C &T-?-?@ T))'23 >>R >>&)19#.Cc3;%+]]7%; NN6+t000vx8#+??7#;v(D/IJ(0*=( c.;%+]]6%: ;.7#';;v#67s$?G<*H<HHH:9H:c$|jdS)Nr)r)rArps r)rlzHTTPConnection._encode_requests~~g&&r6crtj|}|r td|d|jdy)z&Validate a method name for putrequest.z)method can't contain control characters.  (found at least )N)$_contains_disallowed_method_pchar_researchr}group)rArsmatchs r)rjzHTTPConnection._validate_methodsI5;;FC ?zJ'',{{}&7q:; ; r6crtj|}|r td|d|jdy)zValidate a url for putrequest.&URL can't contain control characters. rxryN!_contains_disallowed_url_pchar_rer{r r|)rArtr}s r)rkzHTTPConnection._validate_pathsL288= EcWM005 /@CD D r6crtj|}|r td|d|jdy)z9Validate a host so it doesn't contain control characters.rrxryNr)rArr}s r)rzHTTPConnection._validate_hostsL288> EdXN005 /@CD D r6c|jtk7r tt|dr|j d}t |st d|t|}t|D]r\}}t|dr|j d||<n-t|trt|j d||<t||sct d||dj|}|dz|z}|j|y) zkSend a request header line to the server. For example: h.putheader('Accept', 'text/html') rrzInvalid header name rzInvalid header value s s: N)r rirrr_is_legal_header_namer}r enumeraterrry_is_illegal_header_valuerXrY)rArvaluesr one_valuers r)rmzHTTPConnection.putheader s <.<s s r) getresponsezHTTPConnection.getresponsehs  ??t779"DO" <<< '4??"4<<0 0 ??Q **499doo26,,+@H**499T\\*JH   &&(2 22#DL"" O#+O#     NN  s$ D$=EE$D??EE)NN)NF)FFr)'rGrHrIr`r6rr8 HTTP_PORTr(rPrg staticmethodrrr_GLOBAL_DEFAULT_TIMEOUTrur#rr,r1r@rBr=rr7rYr[rerurlrjrkrrmrrprrrJr6r)rr1sIM!NLIJ11 >#'0N0N $;."1H(  @    !#GJ 4(l16(-}~';DD8 Ge G)-bG$G ,=\=r6rcReZdZdZeZdejddddfd ZfdZ xZ S)HTTPSConnectionz(This class allows communication via SSL.Nr)rrrrcrtt| ||||||t|j}||_y)N)r)rrrurr`_context)rArrrrrrrs r)ruzHTTPSConnection.__init__sB /4 1$g2@ >$($$ $ S Sr6rc eZdZy)rNrGrHrIrJr6r)rrs r6rc eZdZy)rNrrJr6r)rrr6rc eZdZy)r NrrJr6r)r r rr6r ceZdZdZy)rc"|f|_||_yr)argsrk)rArks r)ruzUnknownProtocol.__init__sH  r6NrGrHrIrurJr6r)rrsr6rc eZdZy)r NrrJr6r)r r rr6r c eZdZy)r NrrJr6r)r r rr6r c2eZdZddZdZej Zy)r Nc0|f|_||_||_yr)rpartialexpected)rArrs r)ruzIncompleteRead.__init__sH    r6c|jd|jz}nd}d|jjt|j|fzS)Nz, %i more expectedrxz%s(%i bytes read%s))rrrGr=rrKs r)__repr__zIncompleteRead.__repr__sK == $$t}}4AA$(?(?(+DLL(91(>> >r6r)rGrHrIrurr"__str__rJr6r)r r s!>nnGr6r c eZdZy)r NrrJr6r)r r rr6r c eZdZy)rNrrJr6r)rrrr6rc eZdZy)rNrrJr6r)rrrr6rc eZdZy)rNrrJr6r)rrrr6rceZdZdZy)rc<|s t|}|f|_||_yr)r{rrErs r)ruzBadStatusLine.__init__s:DE  r6NrrJr6r)rrsr6rceZdZdZy)rcBtj|dt|fzy)Nz&got more than %d bytes when reading %s)rrurP)rA line_types r)ruzLineTooLong.__init__s"t%M(0)'<&= >r6NrrJr6r)rrs>r6rceZdZdZy)rcbtj|dtj|g|i|y)Nrx)rruConnectionResetError)rAposkws r)ruzRemoteDisconnected.__init__s*tR(%%d7S7B7r6NrrJr6r)rrs8r6r)r&)Hr email.parserrZ email.messagerIr9rrerrDcollections.abcrR urllib.parser__all__rrrjrrirglobalsupdater: __members__rphraserrPrQcompile fullmatchrr{rrrzrr*rr5r?Messager8rTr`rbBufferedIOBaserrrrrr@ ImportError Exceptionrrr rr r r r rrrrrrrr)vs0r)rsqDL !     #  ,,-#'//"="="D"D"F GQQ[ G   6# #67AA%2::&CDKK%/BJJ/@$A!(2rzz-'@$3 E%%--''<$.9 @)0e2$$eP t t l!&S.S8 NN$% I  =   m  m  M  ]  m  /  .  . M>-> 8-}8 A- Hz)  s H!#H&&H.-H.