diff --git a/Copy-files-between-accounts.ipynb b/Copy-files-between-accounts.ipynb new file mode 100644 index 0000000..2180752 --- /dev/null +++ b/Copy-files-between-accounts.ipynb @@ -0,0 +1,553 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# How to copy files between accounts \n", + "演示在Linux 下 ， 账号间拷贝文件的方式\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "为什么无法直接用cp 复制其他用户账号下的文件呢 ？ 因为没有权限：" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "drwxr-xr-x 14 nscc-gz_jiangli nscc-gz 4096 May 20 15:35 /HOME/nscc-gz_jiangli/tmp\r\n" + ] + } + ], + "source": [ + "ls -ld ~/tmp" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "drwx------ 89 nscc-gz_jiangli nscc-gz_jiangli 12288 May 22 12:51 /HOME/nscc-gz_jiangli\r\n" + ] + } + ], + "source": [ + "ls -ld ~" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "drwxr-x--x 1800 root root 106496 May 23 14:05 /HOME\r\n" + ] + } + ], + "source": [ + "ls -ld /HOME" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ls: cannot open directory /HOME: Permission denied\r\n" + ] + } + ], + "source": [ + "ls /HOME" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "bash: cd: /HOME/nscc_ts_1: Permission denied\r\n" + ] + } + ], + "source": [ + "cd /HOME/nscc_ts_1" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [] + } + ], + "source": [ + "cd /HOME/intel" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "drwxr-xr-x 19 root root 4096 Jun 8 2015 .\r\n" + ] + } + ], + "source": [ + "ls -ld ." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "从前面的演示中，我们可以看到，文件能否访问是受权限控制的。需要你有此文件的 r 权限（可读） ， 以及其所有父文件夹的可执行（访问）权限（x权限）。\n", + "\n", + "如果需要在不同的账号下共享文件，可以考虑以下几种方式 ：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. scp 命令 \n", + "用 scp 命令 和 rsync 命令 在账号间进行账号拷贝是最好的方案之一。不过需要您有两个账号的key 文件。\n", + "这两个命令都是通过 ssh 通道来进行拷贝 ， 命令用法也和 ssh 很像 。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]\r\n", + " [-l limit] [-o ssh_option] [-P port] [-S program]\r\n", + " [[user@]host1:]file1 ... [[user@]host2:]file2\r\n" + ] + } + ], + "source": [ + "scp --help " + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "nscc-gz_kangyouzhong\r\n" + ] + } + ], + "source": [ + "ssh -i ~/keys/id_rsa.ky nscc-gz_kangyouzhong@ln3-gn0 whoami" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "可以看到，通过这个KEY 文件可以成功访问 KY 的账号，那么，scp 命令也没问题。" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\r", + "file.lj 100% 0 0.0KB/s 00:00 \r\n" + ] + } + ], + "source": [ + "scp -i ~/keys/id_rsa.ky ~/file.lj nscc-gz_kangyouzhong@ln3-gn0:~/." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "可以看到文件 ~/file.lj 顺利从我的账号传到了 ky 的账号 " + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\r", + "file.ky 100% 0 0.0KB/s 00:00 \r\n" + ] + } + ], + "source": [ + "scp -i ~/keys/id_rsa.ky nscc-gz_kangyouzhong@ln3-gn0:~/file.ky ~/." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "同样的办法也可以将数据从ky 的账号拷贝到我的账号下。\n", + "\n", + "但是， SCP 命令也有些问题：\n", + "\n", + "* 不支持断点续传\n", + "\n", + "* 对文件夹的传输支持不够好，可能会出现BUG 。\n", + "\n", + "那么替代的解决办法是： rsync 命令 。 当然，对于临时的需求，这种方式简单易用，挺不错的。\n", + "\n", + "## 2. rsync 命令 " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "rsync version 3.0.6 protocol version 30\r\n", + "Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.\r\n", + "Web site: http://rsync.samba.org/\r\n", + "Capabilities:\r\n", + " 64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,\r\n", + " socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,\r\n", + " append, ACLs, xattrs, iconv, symtimes\r\n", + "\r\n", + "rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you\r\n", + "are welcome to redistribute it under certain conditions. See the GNU\r\n", + "General Public Licence for details.\r\n", + "\r\n", + "rsync is a file transfer program capable of efficient remote update\r\n", + "via a fast differencing algorithm.\r\n", + "\r\n", + "Usage: rsync [OPTION]... SRC [SRC]... DEST\r\n", + " or rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST\r\n", + " or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST\r\n", + " or rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST\r\n", + " or rsync [OPTION]... [USER@]HOST:SRC [DEST]\r\n", + " or rsync [OPTION]... [USER@]HOST::SRC [DEST]\r\n", + " or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]\r\n", + "The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect\r\n", + "to an rsync daemon, and require SRC or DEST to start with a module name.\r\n", + "\r\n", + "Options\r\n", + " -v, --verbose increase verbosity\r\n", + " -q, --quiet suppress non-error messages\r\n", + " --no-motd suppress daemon-mode MOTD (see manpage caveat)\r\n", + " -c, --checksum skip based on checksum, not mod-time & size\r\n", + " -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)\r\n", + " --no-OPTION turn off an implied OPTION (e.g. --no-D)\r\n", + " -r, --recursive recurse into directories\r\n", + " -R, --relative use relative path names\r\n", + " --no-implied-dirs don't send implied dirs with --relative\r\n", + " -b, --backup make backups (see --suffix & --backup-dir)\r\n", + " --backup-dir=DIR make backups into hierarchy based in DIR\r\n", + " --suffix=SUFFIX set backup suffix (default ~ w/o --backup-dir)\r\n", + " -u, --update skip files that are newer on the receiver\r\n", + " --inplace update destination files in-place (SEE MAN PAGE)\r\n", + " --append append data onto shorter files\r\n", + " --append-verify like --append, but with old data in file checksum\r\n", + " -d, --dirs transfer directories without recursing\r\n", + " -l, --links copy symlinks as symlinks\r\n", + " -L, --copy-links transform symlink into referent file/dir\r\n", + " --copy-unsafe-links only \"unsafe\" symlinks are transformed\r\n", + " --safe-links ignore symlinks that point outside the source tree\r\n", + " -k, --copy-dirlinks transform symlink to a dir into referent dir\r\n", + " -K, --keep-dirlinks treat symlinked dir on receiver as dir\r\n", + " -H, --hard-links preserve hard links\r\n", + " -p, --perms preserve permissions\r\n", + " -E, --executability preserve the file's executability\r\n", + " --chmod=CHMOD affect file and/or directory permissions\r\n", + " -A, --acls preserve ACLs (implies --perms)\r\n", + " -X, --xattrs preserve extended attributes\r\n", + " -o, --owner preserve owner (super-user only)\r\n", + " -g, --group preserve group\r\n", + " --devices preserve device files (super-user only)\r\n", + " --copy-devices copy device contents as regular file\r\n", + " --specials preserve special files\r\n", + " -D same as --devices --specials\r\n", + " -t, --times preserve modification times\r\n", + " -O, --omit-dir-times omit directories from --times\r\n", + " --super receiver attempts super-user activities\r\n", + " --fake-super store/recover privileged attrs using xattrs\r\n", + " -S, --sparse handle sparse files efficiently\r\n", + " -n, --dry-run perform a trial run with no changes made\r\n", + " -W, --whole-file copy files whole (without delta-xfer algorithm)\r\n", + " -x, --one-file-system don't cross filesystem boundaries\r\n", + " -B, --block-size=SIZE force a fixed checksum block-size\r\n", + " -e, --rsh=COMMAND specify the remote shell to use\r\n", + " --rsync-path=PROGRAM specify the rsync to run on the remote machine\r\n", + " --existing skip creating new files on receiver\r\n", + " --ignore-existing skip updating files that already exist on receiver\r\n", + " --remove-source-files sender removes synchronized files (non-dirs)\r\n", + " --del an alias for --delete-during\r\n", + " --delete delete extraneous files from destination dirs\r\n", + " --delete-before receiver deletes before transfer, not during\r\n", + " --delete-during receiver deletes during transfer (default)\r\n", + " --delete-delay find deletions during, delete after\r\n", + " --delete-after receiver deletes after transfer, not during\r\n", + " --delete-excluded also delete excluded files from destination dirs\r\n", + " --ignore-errors delete even if there are I/O errors\r\n", + " --force force deletion of directories even if not empty\r\n", + " --max-delete=NUM don't delete more than NUM files\r\n", + " --max-size=SIZE don't transfer any file larger than SIZE\r\n", + " --min-size=SIZE don't transfer any file smaller than SIZE\r\n", + " --partial keep partially transferred files\r\n", + " --partial-dir=DIR put a partially transferred file into DIR\r\n", + " --delay-updates put all updated files into place at transfer's end\r\n", + " -m, --prune-empty-dirs prune empty directory chains from the file-list\r\n", + " --numeric-ids don't map uid/gid values by user/group name\r\n", + " --timeout=SECONDS set I/O timeout in seconds\r\n", + " --contimeout=SECONDS set daemon connection timeout in seconds\r\n", + " -I, --ignore-times don't skip files that match in size and mod-time\r\n", + " --size-only skip files that match in size\r\n", + " --modify-window=NUM compare mod-times with reduced accuracy\r\n", + " -T, --temp-dir=DIR create temporary files in directory DIR\r\n", + " -y, --fuzzy find similar file for basis if no dest file\r\n", + " --compare-dest=DIR also compare destination files relative to DIR\r\n", + " --copy-dest=DIR ... and include copies of unchanged files\r\n", + " --link-dest=DIR hardlink to files in DIR when unchanged\r\n", + " -z, --compress compress file data during the transfer\r\n", + " --compress-level=NUM explicitly set compression level\r\n", + " --skip-compress=LIST skip compressing files with a suffix in LIST\r\n", + " -C, --cvs-exclude auto-ignore files the same way CVS does\r\n", + " -f, --filter=RULE add a file-filtering RULE\r\n", + " -F same as --filter='dir-merge /.rsync-filter'\r\n", + " repeated: --filter='- .rsync-filter'\r\n", + " --exclude=PATTERN exclude files matching PATTERN\r\n", + " --exclude-from=FILE read exclude patterns from FILE\r\n", + " --include=PATTERN don't exclude files matching PATTERN\r\n", + " --include-from=FILE read include patterns from FILE\r\n", + " --files-from=FILE read list of source-file names from FILE\r\n", + " -0, --from0 all *-from/filter files are delimited by 0s\r\n", + " -s, --protect-args no space-splitting; only wildcard special-chars\r\n", + " --address=ADDRESS bind address for outgoing socket to daemon\r\n", + " --port=PORT specify double-colon alternate port number\r\n", + " --sockopts=OPTIONS specify custom TCP options\r\n", + " --blocking-io use blocking I/O for the remote shell\r\n", + " --stats give some file-transfer stats\r\n", + " -8, --8-bit-output leave high-bit chars unescaped in output\r\n", + " -h, --human-readable output numbers in a human-readable format\r\n", + " --progress show progress during transfer\r\n", + " -P same as --partial --progress\r\n", + " -i, --itemize-changes output a change-summary for all updates\r\n", + " --out-format=FORMAT output updates using the specified FORMAT\r\n", + " --log-file=FILE log what we're doing to the specified FILE\r\n", + " --log-file-format=FMT log updates using the specified FMT\r\n", + " --password-file=FILE read daemon-access password from FILE\r\n", + " --list-only list the files instead of copying them\r\n", + " --bwlimit=KBPS limit I/O bandwidth; KBytes per second\r\n", + " --write-batch=FILE write a batched update to FILE\r\n", + " --only-write-batch=FILE like --write-batch but w/o updating destination\r\n", + " --read-batch=FILE read a batched update from FILE\r\n", + " --protocol=NUM force an older protocol version to be used\r\n", + " --iconv=CONVERT_SPEC request charset conversion of filenames\r\n", + " -4, --ipv4 prefer IPv4\r\n", + " -6, --ipv6 prefer IPv6\r\n", + " --version print version number\r\n", + "(-h) --help show this help (-h works with no other options)\r\n", + "\r\n", + "Use \"rsync --daemon --help\" to see the daemon-mode command-line options.\r\n", + "Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation.\r\n", + "See http://rsync.samba.org/ for updates, bug reports, and answers\r\n" + ] + } + ], + "source": [ + "rsync --help" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "rsync 是复杂而强大的工具。不过一般来说，下面这种用法可以解决大多数问题：" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [] + } + ], + "source": [ + "rsync -e \"ssh -i /HOME/nscc-gz_jiangli/keys/id_rsa.ky\" ~/file.lj nscc-gz_kangyouzhong@ln3-gn0:~/. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在 -e 里配置建立 ssh 连接需要的特殊参数，这里只是指定了 key 文件，有时候，你可能还需要指定其他的，如用 -p 指定端口。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. 其他临时共享办法" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [] + } + ], + "source": [ + "cp ~/file.lj /tmp/." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [] + } + ], + "source": [ + "chmod 777 /tmp/file.lj" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "如上例所示，将文件拷贝到一个公共文件夹下，并将权限放开，是一个比较简易的文件共享方式。\n", + "\n", + "不过您需要明确这种做法带来的问题：\n", + "\n", + "文件人人可以获取。\n", + "\n", + "此共享文件夹可能会被清理。\n", + "\n", + "如果此共享文件夹是在内存里（如tmp），需要在同一结点才能访问。并且如果文件太大会使系统出现故障。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Bash", + "language": "bash", + "name": "bash" + }, + "language_info": { + "codemirror_mode": "shell", + "file_extension": ".sh", + "mimetype": "text/x-sh", + "name": "bash" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}