-=[ Mr. Bumblebee ]=-
_Indonesia_
:Nc @@ s; d Z d d l m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l m
Z
m Z m Z m Z d d l m Z m Z m Z m Z m Z m Z d d l m Z d d l m Z m Z d d l m Z m Z m Z m! Z! e j" d d
d e# d d
y d d l$ Z$ Wn e% k
rgZ& e e& nU Xd d l' m( Z( m) Z) m* Z* m+ Z+ m, Z, m- Z- m. Z. d d l/ m0 Z0 d d l1 m2 Z2 e3 e$ d d d d f Z4 e4 d d d f k Z5 d e6 f d YZ7 d e6 f d YZ8 d e! f d YZ9 d Z: d S( s6 Implementation of Transport over SFTP, using paramiko.i ( t absolute_importN( t configt debugt errorst urlutils( t
FileExistst
NoSuchFilet TransportErrort LockErrort PathErrort ParamikoNotPresent( t fancy_rename( t muttert warning( t FileFileStreamt
_file_streamst ssht ConnectedTransportt ignores$ integer argument expected, got floatt categoryt modules paramiko.message( t SFTP_FLAG_WRITEt SFTP_FLAG_CREATEt SFTP_FLAG_EXCLt SFTP_FLAG_TRUNCt SFTP_OKt
CMD_HANDLEt CMD_OPEN( t SFTPAttributes( t SFTPFilet __version_info__i i t SFTPLockc B@ s2 e Z d Z d d d d g Z d Z d Z RS( s This fakes a lock in a remote location.
A present lock is indicated just by the existence of a file. This
doesn't work well on all transports and they are only used in
deprecated storage formats.
t patht lock_patht lock_filet transportc C@ s~ d | _ | | _ | d | _ | | _ y( | j | j } | j | | _ Wn' t k
ry t d | j f n Xd S( Ns .write-locks File %r already locked( t NoneR" R R! R# t _remote_patht _sftp_open_exclusiveR R ( t selfR R# t abspath( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt __init__d s
c C@ sU | j s
d S| j j d | _ y | j j | j Wn t f k
rP n Xd S( N( R" t closeR$ R# t deleteR! R ( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt unlockp s
( t __name__t
__module__t __doc__t __slots__R) R, ( ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR Z s t _SFTPReadvHelperc B@ s/ e Z d Z d Z d Z d Z d Z RS( s; A class to help with managing the state of a readv request.i c C@ s% t | | _ | | _ | | _ d S( s4 Create a new readv helper.
:param original_offsets: The original requests given by the caller of
readv()
:param relpath: The name of the file (if known)
:param _report_activity: A Transport._report_activity bound method,
to be called as data arrives.
N( t listt original_offsetst relpatht _report_activity( R' R3 R4 R5 ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR) s c C@ s t | j } t t j | d d d d } g } xl | D]d } | j } | j } xI | d k r t | | j } | j | | f | | 8} | | 7} qX Wq= Wd t
j k r t d | j
t | t | t | n | S( s
Break up the offsets into individual requests over sftp.
The SFTP spec only requires implementers to support 32kB requests. We
could try something larger (openssh supports 64kB), but then we have to
handle requests that fail.
So instead, we just break up our maximum chunks into 32kB chunks, and
asyncronously requests them.
Newer versions of paramiko would do the chunking for us, but we want to
start processing results right away, so we do it ourselves.
t limiti t fudge_factort sftps8 SFTP.readv(%s) %s offsets => %s coalesced => %s requests( t sortedR3 R2 R t _coalesce_offsetst startt lengtht mint _max_request_sizet appendR t debug_flagsR R4 t len( R' t sorted_offsetst coalescedt requestst c_offsetR; t sizet next_size( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt
_get_requests s"
c c@ s | j } t | j } | j \ } } d } d } g } d } g }
t j | j | t j d } x t j | | D]
\ \ } }
} | d k r t
d k r t j | j
| |
t | q n t | |
k r t j | j
| |
t | n | j |
d | d k r=| g } |
} | } nl | | k rc| j | | |
7} nF | d k rd j | } |
j | | f n | } | g } |
} | |
} | | k r | | k r d j | } | 2d } xb | | k rH| | | k rH| | | | !} | | 7} | | 7} | | f V| j \ } } qW| t | k rjg } d } q| | } | g } t | } q q W| j | rd j | } | 2|
j | | f n |
rd t j k r t d t t d |
n x t rt j |
| f } | t |
k rY|
| d | k rY|
| d | } nP | d k r| d 8} | |
| d } |
| d } | | | | !} n d } t | | k rt d | t | f n | | f V| j \ } } qWn d S(
s Request the data from the remote machine, yielding the results.
:param fp: A Paramiko SFTPFile object that supports readv.
:return: Yield the data requested by the original readv caller, one by
one.
i t readt R8 s* SFTP readv left with %d out-of-order bytesc S@ s t | d S( Ni ( RA ( t x( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt <lambda> s i sB We must have miscalulated. We expected %d bytes, but only found %dN( RH t iterR3 t nextR$ t itertoolst chaint readvt repeatt izipt
cur_coalescedR t ShortReadvErrorR4 RA R5 R? t joinR* R R@ R t sumt mapt Truet bisectt bisect_leftt AssertionError( R' t fpRD t offset_itert
cur_offsett cur_sizet input_startt last_endt
buffered_datat buffered_lent data_chunkst data_streamR; R< t datat bufferedt buffered_offsett cur_datat idxt
sub_offset( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt request_and_yield_offsets s %
&
( R- R. R/ R> R) RH Rm ( ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR1 | s
$t
SFTPTransportc B@ s e Z d Z e Z d Z d Z d Z d Z d) d Z
d Z d Z d Z
d Z d
Z d Z d Z d
Z d) d Z d) d Z d) e d) d Z d) e d) d Z d) e d) d Z d Z d) d Z d) d Z d) d Z d e d Z d) d Z d Z d Z! d Z" d Z# d Z$ d Z% d Z& d! Z' d" Z( d# Z) d$ Z* d% Z+ d& Z, d) d' Z- d( Z. RS(* s) Transport implementation for SFTP access.i i i c C@ sJ | j j | j } | j d r1 | d } n | d k rF d } n | S( s{ Return the path to be passed along the sftp protocol for relpath.
:param relpath: is a urlencoded string.
s /~/i s /~RJ ( t _parsed_urlt cloneR t
startswith( R' R4 t remote_path( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR% O s
c C@ s | d k r | j j } n | } t j } | j j } | d k ru t j } | j d | j j | j j
} n | j | j j | | j j | j j
} | | | f f S( s Create a new connection with the provided credentials.
:param credentials: The credentials needed to establish the connection.
:return: The created connection and its associated credentials.
The credentials are only the password as it may have been entered
interactively by the user and may be different from the one provided
in base url at transport creation time.
R N( R$ Ro t passwordR t _get_ssh_vendort userR t AuthenticationConfigt get_usert hostt portt connect_sftp( R' t credentialsRs t vendorRu t autht
connection( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _create_connectiona s c C@ s) | j } | d k r% | j n d S( N( t _get_connectionR$ R* ( R' R~ ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt
disconnect{ s c C@ sA | j } | d k r= | j \ } } | j | | n | S( s( Ensures that a connection is establishedN( R R$ R t _set_connection( R' R~ R{ ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _get_sftp s
c C@ sM y4 | j j | j | | j d d t SWn t k
rH t SXd S( s1
Does the target location exist?
i RI N( R t statR% R5 RY t IOErrort False( R' R4 ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt has s
c C@ s y` | j | } | j j | d d } | j r[ t | d d d k r[ | j n | SWn8 t t j f k
r } | j
| | d d t j n Xd S( sh Get the file at the given relative path.
:param relpath: The relative path to the file
t modet rbt prefetchs : error retrievingt failure_excN(
R% R t filet _do_prefetcht getattrR$ R R t paramikot SSHExceptiont _translate_io_exceptionR t ReadError( R' R4 R t ft e( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt get s !
c C@ sK | j | } z* | j } | j t | d | SWd | j Xd S( NRI ( R RI R5 RA R* ( R' R4 R t bytes( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt get_bytes s c C@ s | s
d Sy | j | } | j j | d d } t | d d } | rb | j | | | Sd t j k r t d t | n | j
| | | SWn/ t t j
f k
r } | j | | d n Xd S( s See Transport.readv()NR R RQ R8 s seek and read %s offsetss : error retrieving( R% R R R R$ t _sftp_readvR R@ R RA t _seek_and_readR R R R ( R' R4 t offsetsR R] RQ R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _readv s c C@ s d S( s See Transport.recommended_page_size().
For SFTP we suggest a large page size to reduce the overhead
introduced by latency.
i@ i i ( ( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt recommended_page_size s c C@ s" t | | | j } | j | S( s Use the readv() member of fp to do async readv.
Then read them using paramiko.readv(). paramiko.readv()
does not support ranges > 64K, so it caps the request size, and
just reads until it gets all the stuff it wants.
( R1 R5 Rm ( R' R] R R4 t helper( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s c C@ s% | j | } | j | | d | S( s
Copy the file-like object into the location.
:param relpath: Location to put the contents, relative to base.
:param f: File-like object.
:param mode: The final mode for the file
R ( R% t _put( R' R4 R R t
final_path( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt put_file s c
C@ s[ d | t j t j t j d d f } | j | d | } t } y y# | j t | j | | } Wn, t
t j f k
r } | j
| | n X| d k r | j j | | n | j t } | j | | | SWnl t k
rV} d d l } t | j y* | s,| j n | j j | Wn
| n X n Xd S( sB Helper function so both put() and copy_abspaths can reuse the codes %s.tmp.%.9f.%d.%di iR N( t timet ost getpidt randomt randintR& R t
set_pipelinedRY t _pumpR R R R R$ R t chmodR* t _rename_and_overwritet Exceptiont tracebackR t
format_exct remove(
R' R( R R t tmp_abspatht foutt closedR< R R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s4
c @ s j | f d } | s8 | d Sy | Wn: t k
r t j j } j | | | n Xd S( Nc @ s d } z y6 j j d d } | j t | Wn/ t j t f k
rp } j | d n X d k r j j n Wd | d k r | j
n Xd S( s3 Try to open the target file, raise error on failureR t wbs : unable to openN( R$ R R R RY R R R R R R* ( R R ( R( R R' t writer( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _open_and_write_file s
( R% R R R t dirnamet _mkdir( R' R4 R R t create_parent_dirt dir_modeR t
parent_dir( ( R( R R' R s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _put_non_atomic_helper s
c @ s8 f d } j | | d | d | d | d S( s Copy the file-like object into the target location.
This function is not strictly safe to use. It is only meant to
be used when you already know that the target does not exist.
It is not safe, because it will open and truncate the remote
file. So there may be a time when the file has invalid contents.
:param relpath: The remote location to put the contents.
:param f: File-like object.
:param mode: Possible access permissions for new file.
None means do not set remote permissions.
:param create_parent_dir: If we cannot create the target file because
the parent directory does not exist, go ahead and
create it, and then try again.
c @ s j | d S( N( R ( R ( R R' ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR N s R R R N( R ( R' R4 R R R R R ( ( R R' s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt put_file_non_atomic< s c @ s5 f d } | j | | d | d | d | d S( Nc @ s | j d S( N( t write( R ( R ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR W s R R R ( R ( R' R4 R R R R R ( ( R s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt put_bytes_non_atomicT s c c@ s t | j d } x | r | j d } | j | } t j | j r xC t | j | D]$ \ } } | j | | d | qd Wq | Vq Wd S( s7 Walk the relative paths of all files in this transport.t .i t /N( R2 t list_dirt popR t S_ISDIRt st_modet enumeratet insert( R' t queueR4 t stt it basename( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt iter_files_recursive] s " c C@ s | d k r d } n | } y | j t | d | j j | | | j d d | d k r | j j | } | d @} | | j d @k r | j d @r t d | d | f n | j j | d | q n Wn5 t j
t f k
r} | j | | d d t
n Xd S(
Ni R i RI i s About to chmod %s over sftp, which will result in its suid or sgid bits being cleared. If you want to preserve those bits, change your environment on the server to use umask 0%03o.R s : unable to mkdirR ( R$ R5 RA R t mkdirt lstatR R
R R R R R R ( R' R( R t
local_modeR R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR j s$
#c C@ s | j | j | d | d S( s% Create a directory at the given path.R N( R R% ( R' R4 R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s c C@ s | j | d | | j | } d } y, | j j | d d } | j t Wn/ t j t f k
r } | j
| | d n X| t | j | <t
| | | S( s See Transport.open_write_stream.RJ R R s : unable to openN( R R% R$ R R R RY R R R R R R( R ( R' R4 R R( t handleR ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt open_write_stream s RJ c C@ s | j | | d t t | d d d k rQ| j d k sL | j d k rh t | t | | n | j d k s | j d j d r t | t | | n | j d k r | | t | | n | j d j d st | d
d t j
k r t j | t | n | j d k r>t j
n t d | j n t | d
d d k r|t d
| j n | d S( s Translate a paramiko or IOError into a friendlier exception.
:param e: The original exception
:param path: The path in question when the error is raised
:param more_info: Extra information that can be included,
such as what was going on
:param failure_exc: Paramiko has the super fun ability to raise completely
opaque errors that just set "e.args = ('Failure',)" with
no more information.
If this parameter is set, it defines the exception
to raise in these cases.
t
raise_generict argss No such file or directorys No such files mkdir failedi s syserr: File existst Failures Directory not empty: t errnos Operation unsupporteds Raising exception with args %ss Raising exception with errno %sN( s No such file or directory( s No such file( s mkdir failed( R ( s Operation unsupported( t _translate_errorR R R$ R R t strRq R R t ENOTEMPTYR t DirectoryNotEmptyt TransportNotPossibleR ( R' R R t more_infoR ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s& c C@ s yp | j | } | j j | d } | d k rO | j j | | n | j } | j | | | SWn/ t t j f k
r } | j
| | d n Xd S( sZ
Append the text in the file-like object into the final
location.
t abs : unable to appendN( R% R R R$ R t tellR R R R R ( R' R4 R R R R t resultR ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt append_file s c C@ sf y, | j j | j | | j | Wn3 t t j f k
ra } | j | | d | n Xd S( s" Rename without special overwritings : unable to rename to %rN( R t renameR% R R R R ( R' t rel_fromt rel_toR ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s c C@ si y/ | j } t | | d | j d | j Wn3 t t j f k
rd } | j | | d | n Xd S( sg Do a fancy rename on the remote server.
Using the implementation provided by osutils.
t rename_funct unlink_funcs : unable to rename to %rN( R R R R R R R R ( R' t abs_fromt abs_toR8 R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s c C@ s2 | j | } | j | } | j | | d S( s3 Move the item at rel_from to the location at rel_toN( R% R ( R' R R t path_fromt path_to( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt move s c C@ s\ | j | } y | j j | Wn/ t t j f k
rW } | j | | d n Xd S( s Delete the item at relpaths : unable to deleteN( R% R R R R R R ( R' R4 R R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR+ s
c C@ s | j S( s, See bzrlib.transport.Transport.external_url.( t base( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt external_url s c C@ s t S( s+ Return True if this store supports listing.( RY ( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt listable s c C@ s | j | } y8 | j j | } | j t t t | d Wn/ t t j f k
rx } | j
| | d n Xg | D] } t j | ^ q S( sC
Return a list of all files at the given location.
RI s : failed to list_dir(
R% R t listdirR5 RW RX RA R R R R R t escape( R' R4 R t entriesR t entry( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s #c C@ s\ | j | } y | j j | SWn/ t t j f k
rW } | j | | d n Xd S( s See Transport.rmdir.s : failed to rmdirN( R% R t rmdirR R R R ( R' R4 R R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s
c C@ s\ | j | } y | j j | SWn/ t t j f k
rW } | j | | d n Xd S( s' Return the stat information for a file.s : unable to statN( R% R R R R R R ( R' R4 R R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR s
c C@ s\ | j | } y | j j | SWn/ t t j f k
rW } | j | | d n Xd S( s See Transport.readlink.s : unable to readlinkN( R% R t readlinkR R R R ( R' R4 R R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR & s
c C@ s yJ | j } | j | | } t | k rI t d | | f | n Wn3 t t j f k
r } | j | | d | n Xd S( s See Transport.symlink.s" %r: unable to create symlink to %rs : unable to create symlink to %rN( R t symlinkR R R R R R ( R' t sourcet link_namet connt sftp_retvalR ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR . s
c C@ s d t f d Y} | | S( s
Lock the given file for shared (read) access.
:return: A lock object, which has an unlock() member function
t BogusLockc B@ s e Z d Z d Z RS( c S@ s
| | _ d S( N( R ( R' R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR) C s c S@ s d S( N( ( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR, E s ( R- R. R) R, ( ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR B s ( t object( R' R4 R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt lock_read< s c C@ s
t | | S( s
Lock the given file for exclusive (write) access.
WARNING: many transports do not support this, so trying avoid using it
:return: A lock object, which has an unlock() member function
( R ( R' R4 ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt
lock_writeI s c
C@ s | j j | } t } | d k r6 | | _ n t t Bt Bt B} yh | j j t
| | | \ } } | t k r t d n | j
} t | j | d d SWn5 t j t f k
r } | j | | d d t n Xd S( s Open a remote path exclusively.
SFTP supports O_EXCL (SFTP_FLAG_EXCL), which fails if
the file already exists. However it does not expose this
at the higher level of SFTPClient.open(), so we have to
sneak away with it.
WARNING: This breaks the SFTPClient abstraction, so it
could easily break against an updated version of paramiko.
:param abspath: The remote absolute path where the file should be opened
:param mode: The mode permissions bits for the new file
s Expected an SFTP handleR is : unable to openR N( R t _adjust_cwdR R$ R R R R R t _requestR R R t
get_stringR R R R R R (
R' R( R R t attrt omodet tt msgR R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyR& V s $ c C@ s t j d k r t St Sd S( Nt win32( t syst platformR RY ( R' ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt _can_roundtrip_unix_modebitsy s N(/ R- R. R/ t _default_do_prefetchR t _max_readv_combinet _bytes_to_read_before_seekR> R% R$ R R R R R R R R R R R R R R R R R R R R R R R R R R+ R R R R R R R R R R& R ( ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyRn : s`
1 *
&
#c C@ s8 d d l m } t | j f t | j f t | j f g S( s. Return the permutations to be used in testing.i ( t stub_sftp( t bzrlib.testsR Rn t SFTPAbsoluteServert SFTPHomeDirServert SFTPSiblingAbsoluteServer( R ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt get_test_permutations s (; R/ t
__future__R RZ R RO R R R R R t warningst bzrlibR R R R t
bzrlib.errorsR R R R R R
t bzrlib.osutilsR t bzrlib.traceR R
t bzrlib.transportR R R R t filterwarningst DeprecationWarningR t ImportErrorR t
paramiko.sftpR R R R R R R t paramiko.sftp_attrR t paramiko.sftp_fileR R t _paramiko_versionR R R R1 Rn R ( ( ( s9 /usr/lib/python2.7/dist-packages/bzrlib/transport/sftp.pyt <module> sD "." 4" I
Copyright © 2017 || Recoded By Mr.Bumblebee