最近豆子公司在转移文件服务器,大概有80T的文件需要传送到云端。豆子直接使用了robocopy进行拷贝。因为历史原因,有些文件夹的权限设置比较奇葩,导致豆子的账户也没有权限去访问,这样的结果就是Robocopy里面可能会有几十百甚至上千个文件夹因为权限问题而无法拷贝。
robocopy 命令例子
1
|
robocopy c:\
source
d:\destination
/E
/w
:30
/r
:3
/log
+:
"c:\temp\log.txt"
/xf
.*
|
那么如果解决这个问题呢?豆子的思路是获取robocopy的日志文件,通过正则提取其中失败的路径,然后通过脚本重新夺取管理员权限,然后重新赋予NTFS的权限。
比如说,下面的robocopy 日志,报错信息如下所示
1
2
3
4
5
6
7
|
New Dir 0\\syd02\Track\.TemporaryItems\folders.1138144168\Cleanup At Startup\.BAH.FftCc\
2017
/11/12
08:58:18 ERROR 5 (0x00000005) Scanning Source Directory \\syd02\Track\.TemporaryItems\folders.1138144168\Cleanup At Startup\.BAH.FftCc\Altitude Business Card Visuals\
Access is denied.
Waiting 180 seconds... Retrying...
2017
/11/12
09:01:18 ERROR 5 (0x00000005) Scanning Source Directory \\syd02\Track\.TemporaryItems\folders.1138144168\Cleanup At Startup\.BAH.FftCc\Altitude Business Card Visuals\
Access is denied.
ERROR: RETRY LIMIT EXCEEDED.
|
提取路径的PowerShell脚本如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
#Read robocopy log files
$contents
=gc C:\temp\tracktransfer.txt -raw
#Regular Expression Pattern
$patt
=
'(\\\\syd02.*\n)Access is denied'
$result
=@()
#Distract the path of broken folders
$contents
|
select-string
-Pattern
$patt
-AllMatches |
foreach
{
$result
+=
$_
.matches.groups |
Where-Object
{
$_
.name
-eq
1}| select -ExpandProperty value
}
#remove duplicated record and blank lines
$result
|sort
-Unique |
foreach
{
$_
.trimend()} | where{
$_
-ne
""} |
set-content
c:\temp\list.txt
|
生成的文本文件如下所示:
list.txt
1
2
3
4
5
6
7
8
9
|
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\1_Acquisition\FY 17\Q1 Campaigns\WBCCAR7498 Q1.1 Campaign\PRODUCTION\
\\syd02\Track\.TemporaryItems\folders.1138144168\Cleanup At Startup\.BAH.FftCc\Altitude Business Card Visuals\
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\1_Acquisition\FY 17\Q1 Campaigns\WBCCAR7487 Q1 OOH - Low Rate\CREATIVE\
\\syd02\Track\.TemporaryItems\folders.1138144168\Cleanup At Startup\.BAH.cj82c\Westpac Brand assets\
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\2_Lifecycle\WBCCAR7602 Additional Cardholder\COPY\
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\2_Lifecycle\WBCCAR7602 Additional Cardholder\PRODUCTION\
\\syd02\Track\CLIENTS\WESTPAC\Westpac CRM\Business\3. COMPLETE\2016\WBCCRM7595_Q4 SME Relationship email\CREATIVE\FINAL\Individual modules\Modules Half Width\
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\1_Acquisition\FY 17\Q2 Campaigns\WBCCAR7516 Microsite Optimisation\FINANCE\
\\syd02\Track\CLIENTS\WESTPAC\Westpac Cards\1_Acquisition\FY 17\Q1 Campaigns\WBCCAR7487 Q1 OOH - Low Rate\FINANCE\
|
因为最近在复习Python,顺手又用Python写了一个同样的功能,效果是一样的
1
2
3
4
5
6
7
8
9
10
11
12
|
fp
=
open
(
'c:/temp/tracktransfer.txt'
)
fp2
=
open
(
'c:/temp/list.txt'
,
'w'
)
contents
=
fp.read()
pat
=
r
'(\\\\syd02.*\n)Access is denied'
ret
=
re.findall(pat,contents)
ret
=
set
(ret)
for
item
in
ret:
print
(item)
fp2.write(item)
print
(
"total number is %d"
%
len
(ret))
fp.close()
fp2.close()
|
-----------------------
然后接下来是重头戏,如何重新夺取权限和赋值?
PowerShell自带的Get-ACL和Set-ACL命令,以及DOS命令 takeown我都试过,老实说,不太好使,后来很幸运地在网上发现了一个第三方的模块 NTFSSecurity,里面的函数完全实现了我需要的功能。
下载地址:
https://gallery.technet.microsoft.com/scriptcenter/1abd77a5-9c0b-4a2b-acef-90dbb2b84e85#content
下载之后,直接解压到对应的Powershell模块路径,然后重启PowerShell ISE就可以自动加载了。
下面是后半截代码,修改目标目录及其子目录文件的所有者和访问权限
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
$a
=
Get-Content
"C:\temp\list.txt"
#For each folder and subfolders, setup the ownership and NTFS permissions
foreach
(
$i
in
$a
)
{
if
(
test-path
$i
)
{
write-host
Taking ownership of Directory
$i
-fore Green
get-item
$i
|
Set-NTFSOwner
-Account
'omnicom\group Australia it access'
get-item
$i
|
add-ntfsaccess
-account
'omnicom\group Australia it access'
-AccessRights FullControl
get-item
$i
|
Add-NTFSAccess
-Account
'omnicom\sydney track all staff'
-AccessRights modify
$items
= @()
$items
=
$null
$path
=
$null
#if need to setup all subfolders, we can use -recusrse in the following cmdlet.
$items
=
get-childitem
$i
-force
foreach
(
$item
in
$items
)
{
$path
=
$item
.FullName
Write-Host
...Adding AdminGroup to
$path
-Fore Green
Get-Item
-force
$path
|
Set-NTFSOwner
-Account
'omnicom\group Australia it access'
get-item
$i
|
Add-NTFSAccess
-account
'omnicom\group Australia it access'
-AccessRights FullControl
get-item
$i
|
Add-NTFSAccess
-Account
'omnicom\sydney track all staff'
-AccessRights modify
}
}
}
|
最后的效果是
这样子,如果RoboCopy出了什么权限方面的问题,我通过这个脚本就可以很容易的解决了,一次不行,修复之后再跑一次Robocopy很快就可以把丢失的文件拷贝回来了~