Tiny并行计算框架之复杂示例

简介:

问题来源

非常感谢@doctorwho的问题:

假如职业介绍所来了一批生产汽车的工作,假设生产一辆汽车任务是这样的:搭好底盘、拧4个轮胎、安装发动机、安装4个座椅、再装4个车门、最后安装顶棚。之间有的任务是可以并行计算的(比如拧4个轮胎,安装发动机和安装座椅),有的任务有前置任务(比如先装好座椅,才能装车门和顶棚)。让两组包工头组织两种类型的工作:将工人分成两种类型,即可并行计算的放在同一组内,由职业介绍所来控制A组包工头做完的任务交给B组包工头。中间环节的半成品保存到Warehouse中,是这样使用TINY框架来生产汽车么?

接下来,我就用Tiny并行计算框架来展示一下这个示例,在编写示例的时候,发现了一个BUG,这也充分体现了开源的精神与价值,再次感谢@doctorwho。

问题分析

doctorwho的问题还是比较复杂的,但是实际上道理是一样的,因此我把问题简化成下面的过程

第一步:构建底盘

第二步:并行进行安装引擎,座位和轮胎

第三步:并行进行安装门及车顶

由于我和doctorwho都不是造车行家,因此就不用纠结这么造是不是合理了,假设这么做就是合理的。

代码实现

按我前面说的过程,工人是必须要有的,因此我们首先构建工人:

第一步的底盘构建工人

?
1
2
3
4
5
6
7
8
9
10
11
12
13
public class StepFirstWorker extends AbstractWorker {
     public StepFirstWorker() throws RemoteException {
         super ( "first" );
     }
 
     @Override
     protected Warehouse doWork(Work work) throws RemoteException {
         System.out.println(String.format( "%s 构建底盘完成." , work.getInputWarehouse().get( "carType" )));
         Warehouse outputWarehouse = work.getInputWarehouse();
         outputWarehouse.put( "baseInfo" , "something about baseInfo" );
         return outputWarehouse;
     }
}

由于第二步工人有好几个类型,因此再搞个第二步抽象工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public abstract class StepThirdWorker extends AbstractWorker {
    public StepThirdWorker() throws RemoteException {
        super ( "third" );
    }
 
 
    protected boolean acceptMyWork(Work work) {
        String workClass = work.getInputWarehouse().get( "class" );
        if (workClass != null ) {
            return true ;
        }
        return false ;
    }
    protected Warehouse doMyWork(Work work) throws RemoteException {
        System.out.println(String.format( "Base:%s " , work.getInputWarehouse().get( "baseInfo" )));
        System.out.println(String.format( "%s is Ok" , work.getInputWarehouse().get( "class" )));
        return work.getInputWarehouse();
    }
}

接下来构建第二步的引擎工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public class StepSecondEngineWorker extends StepSecondWorker {
 
 
  public static final String ENGINE = "engine" ;
 
 
  public StepSecondEngineWorker() throws RemoteException {
  super ();
  }
 
 
  public boolean acceptWork(Work work) {
  return acceptMyWork(work);
  }
 
 
  protected Warehouse doWork(Work work) throws RemoteException {
  return super .doMyWork(work);
  }
}

第二步的座位工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
public class StepSecondSeatWorker extends StepSecondWorker {
 
 
    public static final String SEAT = "seat" ;
 
 
    public StepSecondSeatWorker() throws RemoteException {
        super ();
    }
    public boolean acceptWork(Work work) {
        return acceptMyWork(work);
    }
    protected Warehouse doWork(Work work) throws RemoteException {
        return super .doMyWork(work);
    }
}
 
  <div>
 
 
  </div>

第二步的轮胎工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public class StepSecondTyreWorker extends StepSecondWorker {
  public static final String TYRE = "tyre" ;
 
 
  public StepSecondTyreWorker() throws RemoteException {
  super ();
  }
 
 
  public boolean acceptWork(Work work) {
  return acceptMyWork(work);
  }
 
 
  protected Warehouse doWork(Work work) throws RemoteException {
  return super .doMyWork(work);
  }
}

同理,第三步也是大同小异的。

第三步的抽象工人类:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public abstract class StepThirdWorker extends AbstractWorker {
    public StepThirdWorker() throws RemoteException {
        super ( "third" );
    }
 
 
    protected boolean acceptMyWork(Work work) {
        String workClass = work.getInputWarehouse().get( "class" );
        if (workClass != null ) {
            return true ;
        }
        return false ;
    }
    protected Warehouse doMyWork(Work work) throws RemoteException {
        System.out.println(String.format( "Base:%s " , work.getInputWarehouse().get( "baseInfo" )));
        System.out.println(String.format( "%s is Ok" , work.getInputWarehouse().get( "class" )));
        return work.getInputWarehouse();
    }
}

第三步的车门工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public class StepThirdDoorWorker extends StepThirdWorker {
 
 
    public static final String DOOR = "door" ;
 
 
    public StepThirdDoorWorker() throws RemoteException {
        super ();
    }
    public boolean acceptWork(Work work) {
        return acceptMyWork(work);
    }
    @Override
    protected Warehouse doWork(Work work) throws RemoteException {
        return super .doMyWork(work);
    }
}

第三步的车顶工人:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
public class StepThirdRoofWorker extends StepThirdWorker {
 
 
    public static final String ROOF = "roof" ;
 
 
    public StepThirdRoofWorker() throws RemoteException {
        super ();
    }
    public boolean acceptWork(Work work) {
        return acceptMyWork(work);
    }
    protected Warehouse doWork(Work work) throws RemoteException {
        return super .doMyWork(work);
    }
}

以上就把工人都构建好了,我们前面也说过,如果要进行任务分解,是必须要构建任务分解合并器的,这里简单起见,只实现任务分解了。

第二部的任务分解:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
public class SecondWorkSplitter implements WorkSplitter {
     public List<Warehouse> split(Work work, List<Worker> workers) throws RemoteException {
         List<Warehouse> list = new ArrayList<Warehouse>();
         list.add(getWareHouse(work.getInputWarehouse(), "engine" ));
         list.add(getWareHouse(work.getInputWarehouse(), "seat" ));
         list.add(getWareHouse(work.getInputWarehouse(), "seat" ));
         list.add(getWareHouse(work.getInputWarehouse(), "seat" ));
         list.add(getWareHouse(work.getInputWarehouse(), "seat" ));
         list.add(getWareHouse(work.getInputWarehouse(), "tyre" ));
         list.add(getWareHouse(work.getInputWarehouse(), "tyre" ));
         list.add(getWareHouse(work.getInputWarehouse(), "tyre" ));
         list.add(getWareHouse(work.getInputWarehouse(), "tyre" ));
         return list;
     }
 
     private Warehouse getWareHouse(Warehouse inputWarehouse, String stepClass) {
         Warehouse warehouse = new WarehouseDefault();
         warehouse.put( "class" , stepClass);
         warehouse.putSubWarehouse(inputWarehouse);
         return warehouse;
     }
}

从上面可以看到,构建了一个引擎的仓库,4个座位仓库,4个轮胎仓库。呵呵,既然能并行,为啥不让他做得更快些?

接下来是第三步的任务分解器:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public class ThirdWorkSplitter implements WorkSplitter {
     public List<Warehouse> split(Work work, List<Worker> workers) throws RemoteException {
         List<Warehouse> list = new ArrayList<Warehouse>();
         list.add(getWareHouse(work.getInputWarehouse(), "door" ));
         list.add(getWareHouse(work.getInputWarehouse(), "door" ));
         list.add(getWareHouse(work.getInputWarehouse(), "door" ));
         list.add(getWareHouse(work.getInputWarehouse(), "door" ));
         list.add(getWareHouse(work.getInputWarehouse(), "roof" ));
         return list;
     }
 
     private Warehouse getWareHouse(Warehouse inputWarehouse, String stepClass) {
         Warehouse warehouse = new WarehouseDefault();
         warehouse.put( "class" , stepClass);
         warehouse.putSubWarehouse(inputWarehouse);
         return warehouse;
     }
}

从上面可以看到,第三部构建了4个门仓库一个车顶仓库,同样的,可以让4个工人同时装门。

上面就把所有的准备工作都做好了,接下来就是测试方法了:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
public class Test {
     public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
         JobCenter jobCenter = new JobCenterLocal();
 
 
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepFirstWorker());
         }
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepSecondTyreWorker());
         }
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepSecondSeatWorker());
         }
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepSecondEngineWorker());
         }
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepThirdDoorWorker());
         }
         for ( int i = 0 ; i < 5 ; i++) {
             jobCenter.registerWorker( new StepThirdRoofWorker());
         }
 
         jobCenter.registerForeman( new ForemanSelectOneWorker( "first" ));
      <span></span> <span></span>jobCenter.registerForeman( new ForemanSelectAllWorker( "second" ,
  new SecondWorkSplitter()));
  jobCenter.registerForeman( new ForemanSelectAllWorker( "third" , new ThirdWorkSplitter()));
 
 
  Warehouse inputWarehouse = new WarehouseDefault();
  inputWarehouse.put( "class" , "car" );
  inputWarehouse.put( "carType" , "普桑" );
  WorkDefault work = new WorkDefault( "first" , inputWarehouse);
  work.setForemanType( "first" );
  WorkDefault work2 = new WorkDefault( "second" );
  work2.setForemanType( "second" );
  WorkDefault work3 = new WorkDefault( "third" );
  work3.setForemanType( "third" );
  work.setNextWork(work2).setNextWork(work3);
 
 
  Warehouse warehouse = jobCenter.doWork(work);
 
 
  jobCenter.stop();
 
     }
}

呵呵,工人各加了5个,然后注册了三个工头,第一步的工头是随便挑一个工人类型的,第二步和第三步是挑所有工人的,同时还指定了任务分解器。

接下来就构建了一个工作,造一个高端大气上档次的普桑汽车,然后告诉职业介绍所说给我造就可以了。

下面是造车的过程,我把日志也贴上来了:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
普桑 构建底盘完成.
- 234  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:4af96b81d14a4954a6b649308d444e4c,type:second>运行开始,线程数 9 ...
- 234  [id:4af96b81d14a4954a6b649308d444e4c,type:second-a763f156ffd74b5db285198d2498edcf] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-a763f156ffd74b5db285198d2498edcf>运行开始...
- 234  [id:4af96b81d14a4954a6b649308d444e4c,type:second-c2d2fb38ef6c4509b3a39b3e7d5c1d61] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-c2d2fb38ef6c4509b3a39b3e7d5c1d61>运行开始...
- 234  [id:4af96b81d14a4954a6b649308d444e4c,type:second-d624ea0a6df3409c80df6b97ab3c813b] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-d624ea0a6df3409c80df6b97ab3c813b>运行开始...
- 235  [id:4af96b81d14a4954a6b649308d444e4c,type:second-abdb57f0641a4727a9efa744d07cf2d1] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-abdb57f0641a4727a9efa744d07cf2d1>运行开始...
- 236  [id:4af96b81d14a4954a6b649308d444e4c,type:second-d6f7074f6c4a4b12bd37ec5f5c11aff8] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-d6f7074f6c4a4b12bd37ec5f5c11aff8>运行开始...
- 237  [id:4af96b81d14a4954a6b649308d444e4c,type:second-04db3f945b804500a2bbe2b9aabdce3b] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-04db3f945b804500a2bbe2b9aabdce3b>运行开始...
Base:something about baseInfo
engine is Ok
Base:something about baseInfo
seat is Ok
- 245  [id:4af96b81d14a4954a6b649308d444e4c,type:second-a763f156ffd74b5db285198d2498edcf] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-a763f156ffd74b5db285198d2498edcf>运行结束
- 246  [id:4af96b81d14a4954a6b649308d444e4c,type:second-f3efba2dc7804c6cbcd5a25f42fdc177] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-f3efba2dc7804c6cbcd5a25f42fdc177>运行开始...
- 246  [id:4af96b81d14a4954a6b649308d444e4c,type:second-abdb57f0641a4727a9efa744d07cf2d1] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-abdb57f0641a4727a9efa744d07cf2d1>运行结束
Base:something about baseInfo
seat is Ok
- 248  [id:4af96b81d14a4954a6b649308d444e4c,type:second-8c3b9359bcfa4de7b6e0492daab0d73a] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-8c3b9359bcfa4de7b6e0492daab0d73a>运行开始...
Base:something about baseInfo
tyre is Ok
- 250  [id:4af96b81d14a4954a6b649308d444e4c,type:second-f3efba2dc7804c6cbcd5a25f42fdc177] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-f3efba2dc7804c6cbcd5a25f42fdc177>运行结束
- 250  [id:4af96b81d14a4954a6b649308d444e4c,type:second-c2d2fb38ef6c4509b3a39b3e7d5c1d61] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-c2d2fb38ef6c4509b3a39b3e7d5c1d61>运行结束
Base:something about baseInfo
seat is Ok
- 252  [id:4af96b81d14a4954a6b649308d444e4c,type:second-869b573e226046aca8ad30765f1f300c] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-869b573e226046aca8ad30765f1f300c>运行开始...
- 253  [id:4af96b81d14a4954a6b649308d444e4c,type:second-8c3b9359bcfa4de7b6e0492daab0d73a] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-8c3b9359bcfa4de7b6e0492daab0d73a>运行结束
Base:something about baseInfo
seat is Ok
Base:something about baseInfo
tyre is Ok
- 257  [id:4af96b81d14a4954a6b649308d444e4c,type:second-d624ea0a6df3409c80df6b97ab3c813b] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-d624ea0a6df3409c80df6b97ab3c813b>运行结束
- 258  [id:4af96b81d14a4954a6b649308d444e4c,type:second-04db3f945b804500a2bbe2b9aabdce3b] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-04db3f945b804500a2bbe2b9aabdce3b>运行结束
Base:something about baseInfo
tyre is Ok
Base:something about baseInfo
tyre is Ok
- 262  [id:4af96b81d14a4954a6b649308d444e4c,type:second-869b573e226046aca8ad30765f1f300c] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-869b573e226046aca8ad30765f1f300c>运行结束
- 264  [id:4af96b81d14a4954a6b649308d444e4c,type:second-d6f7074f6c4a4b12bd37ec5f5c11aff8] INFO   - 线程<id:4af96b81d14a4954a6b649308d444e4c,type:second-d6f7074f6c4a4b12bd37ec5f5c11aff8>运行结束
- 264  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:4af96b81d14a4954a6b649308d444e4c,type:second>运行结束, 用时:30ms
- 333  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third>运行开始,线程数 5 ...
- 334  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-ca58e9b733514c668a224875417c9d26] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-ca58e9b733514c668a224875417c9d26>运行开始...
- 334  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-17debef817e34c49996a2c38840f3de2] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-17debef817e34c49996a2c38840f3de2>运行开始...
- 334  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-af0e914cce89480987c6184a885770d5] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-af0e914cce89480987c6184a885770d5>运行开始...
- 334  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-3f7707cb2a224d3d8844a09271b24a07] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-3f7707cb2a224d3d8844a09271b24a07>运行开始...
Base:something about baseInfo
door is Ok
Base:something about baseInfo
door is Ok
Base:something about baseInfo
- 338  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-ca58e9b733514c668a224875417c9d26] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-ca58e9b733514c668a224875417c9d26>运行结束
door is Ok
- 339  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-cfccfb37ebda4279b7552f7e060b2ddb] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-cfccfb37ebda4279b7552f7e060b2ddb>运行开始...
- 338  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-17debef817e34c49996a2c38840f3de2] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-17debef817e34c49996a2c38840f3de2>运行结束
Base:something about baseInfo
door is Ok
Base:something about baseInfo
- 340  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-3f7707cb2a224d3d8844a09271b24a07] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-3f7707cb2a224d3d8844a09271b24a07>运行结束
roof is Ok
- 340  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-af0e914cce89480987c6184a885770d5] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-af0e914cce89480987c6184a885770d5>运行结束
- 342  [id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-cfccfb37ebda4279b7552f7e060b2ddb] INFO   - 线程<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third-cfccfb37ebda4279b7552f7e060b2ddb>运行结束
- 343  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third>运行结束, 用时:10ms

从上面的日志可以看出:

由于第一步工作是挑一个单干的,因此是没有启用线程组的

第二步同时有9个线程干活:

?
1
2
3
- 234  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:4af96b81d14a4954a6b649308d444e4c,type:second>运行开始,线程数 9 ...
...
- 264  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:4af96b81d14a4954a6b649308d444e4c,type:second>运行结束, 用时:30ms

第三步同时有5个线程干活:

?
1
2
3
- 333  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third>运行开始,线程数 5 ...
...
- 343  [RMI TCP Connection( 1 )- 192.168 . 84.73 ] INFO   - 线程组<id:9b0de678632d4fb8b87ae9db4b6436f8,type:third>运行结束, 用时:10ms

总结:

Tiny并行计算框架确实是可以方便的解决各种复杂并行计算的问题。

相关文章
|
7月前
|
机器学习/深度学习 数据可视化 PyTorch
TensorFlow与PyTorch框架的深入对比:特性、优势与应用场景
【5月更文挑战第4天】本文对比了深度学习主流框架TensorFlow和PyTorch的特性、优势及应用场景。TensorFlow以其静态计算图、高性能及TensorBoard可视化工具适合大规模数据处理和复杂模型,但学习曲线较陡峭。PyTorch则以动态计算图、易用性和灵活性见长,便于研究和原型开发,但在性能和部署上有局限。选择框架应根据具体需求和场景。
|
7月前
|
机器学习/深度学习 并行计算 PyTorch
【多GPU炼丹-绝对有用】PyTorch多GPU并行训练:深度解析与实战代码指南
本文介绍了PyTorch中利用多GPU进行深度学习的三种策略:数据并行、模型并行和两者结合。通过`DataParallel`实现数据拆分、模型不拆分,将数据批次在不同GPU上处理;数据不拆分、模型拆分则将模型组件分配到不同GPU,适用于复杂模型;数据和模型都拆分,适合大型模型,使用`DistributedDataParallel`结合`torch.distributed`进行分布式训练。代码示例展示了如何在实践中应用这些策略。
2031 2
【多GPU炼丹-绝对有用】PyTorch多GPU并行训练:深度解析与实战代码指南
|
存储 自然语言处理 并行计算
PyTorch 新库 TorchMultimodal 使用说明:将多模态通用模型 FLAVA 扩展到 100 亿参数
PyTorch 新库 TorchMultimodal 使用说明:将多模态通用模型 FLAVA 扩展到 100 亿参数
273 0
|
机器学习/深度学习 存储 并行计算
10个你一定要知道的Pytorch特性
10个你一定要知道的Pytorch特性
179 0
|
机器学习/深度学习 存储 人工智能
从Tensorflow模型文件中解析并显示网络结构图(pb模型篇)
从Tensorflow模型文件中解析并显示网络结构图(pb模型篇)
从Tensorflow模型文件中解析并显示网络结构图(pb模型篇)
|
机器学习/深度学习 缓存 人工智能
Julia开源新框架SimpleChain:小型神经网络速度比PyTorch快5倍!
Julia开源新框架SimpleChain:小型神经网络速度比PyTorch快5倍!
207 0
|
物联网 TensorFlow 算法框架/工具
联邦学习 (FL) 中常见的3种模型聚合方法的 Tensorflow 示例
联合学习 (FL) 是一种出色的 ML 方法,它使多个设备(例如物联网 (IoT) 设备)或计算机能够在模型训练完成时进行协作,而无需共享它们的数据。
393 0
联邦学习 (FL) 中常见的3种模型聚合方法的 Tensorflow 示例
|
TensorFlow 算法框架/工具
TF之TFOD-API:基于tensorflow框架利用TFOD-API脚本文件将YoloV3训练好的.ckpt模型文件转换为推理时采用的.pb文件
TF之TFOD-API:基于tensorflow框架利用TFOD-API脚本文件将YoloV3训练好的.ckpt模型文件转换为推理时采用的.pb文件
TF之TFOD-API:基于tensorflow框架利用TFOD-API脚本文件将YoloV3训练好的.ckpt模型文件转换为推理时采用的.pb文件
|
消息中间件 并行计算 算法
用于并行图处理的 C++ 库libgrape-lite
libgrape-lite是阿里巴巴的一个 C++ 库,用于并行图处理。它与先前系统的不同之处在于,它能够通过遵循GRAPE的PIE编程模型将顺序图算法作为一个整体进行并行化。
用于并行图处理的 C++ 库libgrape-lite
|
TensorFlow 算法框架/工具 Python
【TensorFlow重大升级】自动将Python代码转为TF Graph,大幅简化动态图处理!
TensorFlow发布重大功能改进AutoGraph,能自动将Python代码转换为TensorFlow Graph,TF动态图处理速度大幅提升!
14666 0