Realtime_Multi-Person_Pose_Estimation

View license
Python
跨平台
2021-11-17
首席测试

Realtime Multi-Person Pose Estimation

By Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh.

Introduction

Code repo for winning 2016 MSCOCO Keypoints Challenge, 2016 ECCV Best Demo Award, and 2017 CVPR Oral paper.

Watch our video result in YouTube or our website.

We present a bottom-up approach for realtime multi-person pose estimation, without using any person detector. For more details, refer to our CVPR'17 paper, our oral presentation video recording at CVPR 2017 or our presentation slides at ILSVRC and COCO workshop 2016.

This project is licensed under the terms of the license.

Other Implementations

Thank you all for the efforts for the reimplementation! If you have new implementation and want to share with others, feel free to make a pull request or email me!

Contents

  1. Testing
  2. Training
  3. Citation

Testing

C++ (realtime version, for demo purpose)

  • Please use OpenPose, now it can run in CPU/ GPU and windows /Ubuntu.
  • Three input options: images, video, webcam

Matlab (slower, for COCO evaluation)

  • Compatible with general Caffe. Compile matcaffe.
  • Run cd testing; get_model.sh to retrieve our latest MSCOCO model from our web server.
  • Change the caffepath in the config.m and run demo.m for an example usage.

Python

  • cd testing/python
  • ipython notebook
  • Open demo.ipynb and execute the code

Training

Network Architecture

Teaser?

Training Steps

  • Run cd training; bash getData.sh to obtain the COCO images in dataset/COCO/images/, keypoints annotations in dataset/COCO/annotations/ and COCO official toolbox in dataset/COCO/coco/.
  • Run getANNO.m in matlab to convert the annotation format from json to mat in dataset/COCO/mat/.
  • Run genCOCOMask.m in matlab to obatin the mask images for unlabeled person. You can use 'parfor' in matlab to speed up the code.
  • Run genJSON('COCO') to generate a json file in dataset/COCO/json/ folder. The json files contain raw informations needed for training.
  • Run python genLMDB.py to generate your LMDB. (You can also download our LMDB for the COCO dataset (189GB file) by: bash get_lmdb.sh)
  • Download our modified caffe: caffe_train. Compile pycaffe. It will be merged with caffe_rtpose (for testing) soon.
  • Run python setLayers.py --exp 1 to generate the prototxt and shell file for training.
  • Download VGG-19 model, we use it to initialize the first 10 layers for training.
  • Run bash train_pose.sh 0,1 (generated by setLayers.py) to start the training with two gpus.

Citation

Please cite the paper in your publications if it helps your research:

@inproceedings{cao2017realtime,
  author = {Zhe Cao and Tomas Simon and Shih-En Wei and Yaser Sheikh},
  booktitle = {CVPR},
  title = {Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields},
  year = {2017}
  }
  
@inproceedings{wei2016cpm,
  author = {Shih-En Wei and Varun Ramakrishna and Takeo Kanade and Yaser Sheikh},
  booktitle = {CVPR},
  title = {Convolutional pose machines},
  year = {2016}
  }
加载中

暂无资讯

暂无问答

A 2020 Human Pose Estimation Review (Part3:2D Bottom-Up Multi-Person Pose Estimation)

承接之前的博文: A 2020 Human Pose Estimation Review (Part1:2D Single Person) A 2020 Human Pose Estimation Review (Part2:2D Top-Down Multi-Person Pose Estimation) 目录 1 ...

2020/08/21 15:22
41
0
OpenPose论文笔记《Realtime Multi-Person 2D Human Pose Estimation using Part Affinity Fields》

OpenPose最新论文《Realtime Multi-Person 2D Human Pose Estimation using Part Affinity Fields》笔记 摘要 能有效检测图像中多个人的2D姿态。使用PAFs (Part Affinity Fields),来学习关键...

2018/07/19 14:16
44
0
Towards Accurate Multi-person Pose Estimation in the Wild 论文阅读

论文概况 论文名:Towards Accurate Multi-person Pose Estimation in the Wild 作者(第一作者)及单位:George Papandreou, 谷歌 发表期刊/会议:CVPR2016 被引次数(截止到发博日期,以谷歌...

2018/07/17 22:12
7
0
『关键点检测』CPN:Cascaded Pyramid Network for Multi-Person Pose Estimation

论文连接 网络简介 face++2017年coco keypoint benchmark 数据集冠军的文章,发表于CVPR201 1 提出了一种金字塔型的串接模型,即CPN(cascaded pyramid network),这个模型能够同时兼顾人体...

2018/12/18 16:28
50
0
《Cascaded Pyramid Network for Multi-Person Pose Estimation》论文阅读及复现笔记

一、PipeLine 要点 TopDown + GlobalNet + RefineNet 二、Motivation 通过提高对难以识别的关键点的识别准确率,来提升总体识别准确率。 方法:1.refineNet中提升感受野 2. 在线难例挖掘 三、...

2019/10/28 10:14
2
0
human pose estimation

2D Pose estimation主要面临的困难:遮挡、复杂背景、光照、真实世界的复杂姿态、人的尺度不一、拍摄角度不固定等。 单人姿态估计 传统方法:基于Pictorial Structures, DPM ▪ 基于深度学习...

2019/02/21 15:24
15
0
【人体姿态估计2】Real-time Multi-person 2d pose estimation using part affinity fields_2017

文章目录 1、Introduction 2、Method 2.1 Simultaneous Detection and Association 2.2 Confidence Maps for part detection 2.3 Part Affinity Fields for Part Association 2.4 Multi-pers...

2020/04/06 09:41
52
0
Social Grouping for Multi-Target Tracking and Head Pose Estimation in Video(翻译)

0 - ABSTRACT   许多计算机任务在缺少上下文信息的情况下的处理会更加困难。例如,在多相机跟踪任务下,行人可能在不同照相机下面因为有这不同的姿势和灯光条件而看起来很不一样。类似地,...

2018/08/29 11:44
2
0
Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark阅读笔记

Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark阅读笔记 这篇论文主要是两个贡献:LIP数据集与JPPNet网络。(论文说自己有三大共享,还有一个贡献是探...

2018/06/29 19:47
81
0
[ICRA 2019]Multi-Task Template Matching for Object Detection, Segmentation and Pose Estimation Us...

简介 本文作者提出新的框架(MTTM),使用模板匹配来完成多个任务,从深度图的模板上找到目标物体,通过比较模板特征图与场景特征图来预测分割mask和模板与检测物体之间的位姿变换。作者提出的...

2019/10/20 14:39
10
0

没有更多内容

加载失败,请刷新页面

返回顶部
顶部