南开大学移动与飞行机器人实验室隶属于南开大学人工智能学院机器人与信息自动化研究所，天津市智能机器人技术重点实验室。主要围绕移动与飞行机器人的运动规划、镇定与跟踪控制、激光与视觉SLAM等问题展开研究。实验室具有各式移动机器人，包括主动偏心式全方位移动机器人(NK-OMNI I)、差分式移动机器人(P3-DX)、 各类旋翼飞行器等。

课题组主要研究方向包括：

1）智能感知：包括SLAM、机器人定位、场景理解；

2）运动规划与伺服控制：包括时间最优规划、动态路径与速度规划、视觉伺服控制；

3）自主与遥操作智能融合：包括软硬件设计、人机交互技术、鲁棒高效规划、柔顺控制；

4）基于强化学习的移动机器人对抗博弈技术；

5）家庭与公共服务、青藏高原与极地科考、灾难环境救援、地下空间探索等机器人应用。

课题组详细介绍请参考：http://www.xuebozhang.net/

Pioneer3-DX	Summit XL HL	自研平台 NK-OMNI II	各类自研旋翼飞行器

单线激光	多线激光	力反馈设备	深度相机

1. 在研项目：
2. 2019.10-2022.09，智能移动机器人，天津市杰出青年科学基金（19JCJQJC62100），项目负责人，100万元；
3. 2019.04-2022.03，移动终端交互式全方位机器人鲁棒SLAM 与运动规划，天津市自然科学基金面上项目(19JCYBJC18500)，项目负责人，10万元；

1. 2019.06-2022.05， 面向高海拔科考环境的机器人移动与作业技术研究，“智能机器人”国家重点研发计划课题（2018YFB1307503），课题负责人，334万元；
2. 2017.01-2020.12，基于人类行为模拟的机器人智能控制关键技术与应用研究，国家自然科学基金委员会-深圳市人民政府机器人基础研究中心联合基金重点项目(U1613210)，子课题负责人，直接经费85.8万元/260万元。

1. 已完成项目：
2. 2016.01-2019.12，基于视觉与激光的无人平台环境感知与自主导航共性理论方法与应用研究，南开大学百名青年学科带头人项目，项目负责人，50万；(已结题)
3. 2016.01-2019.12，面向场景监控的全方位移动机器人最短时间覆盖，国家自然科学基金面上项目(61573195)，项目负责人，直接经费65万；(已结题)
4. 2013.01-2015.12，基于人类驾驶行为的移动机器人无标定优化视觉伺服方法研究，国家自然科学基金青年科学基金(61203333)，项目负责人，25万；(已结题)
5. 2013.01-2015.12，移动机器人无期望图像视觉伺服方法研究，高等学校博士学科点专项科研基金(新教师类)(20120031120040)，项目负责人，4万；(已结题)
6. 2013.04-2016.03，基于主动视觉与轨迹预测的机器人目标拦截方法，天津市应用基础与前沿技术研究计划青年项目(13JCQNJC03200), 项目负责人，6万；(已结题)
7. 2011.09-2013.06，基于自标定技术的云台摄像机目标分类与跟踪控制策略研究，中央高校基本科研业务费，项目负责人，15万；(已结题)
8. 2019.10-2019.11，液压装卸臂对位电子裁判系统，横向课题，项目负责人；(已结题)
9. 2017.02-2017.06，花卉视觉分级，横向课题，项目负责人。(已结题)

国际期刊(International Journals)

1. C. Li, B. Li*, R. Wang, and X. Zhang. A survey on visual servoing for wheeled mobile robots, International Journal of Intelligent Robotics and Applications, Springer, 2021, accepted.
2. C. Li, X. Zhang*,  H. Gao, R. Wang, Y. Fang,  Bridging the gap between visual servoing and visual SLAM: a novel integrated interactive framework, IEEE Transactions on Automation Science and Engineering, 2021, Online DOI: 10.1109/TASE.2021.3067792.
3. H. Y. Liu, R. Song*, X. Zhang, Point clouds segmentation based on Euclidean clustering and multi-plane extraction in rugged field,  Measurement Science and Technology, 2021, accepted.
4. H. Wang, S. Zhang, X.Y. Zhang, X. Zhang*, and J. Liu, Near-optimal 3-D visual coverage for quadrotor UAVs under photogrammetric constraints, IEEE Transactions on Industrial Electronics, Online DOI: 10.1109/TIE.2021.3060643
5. R. Wang, X. Zhang*,  Y. Fang, and B. Li, Virtual-goal-guided RRT for visual servoing of mobile robots with FOV constraint, IEEE Transactions  on Systems, Man and Cybernetics: Systems, Online DOI: 10.1109/TSMC.2020.3044347
6. J. Jiang, J. Yuan, X.T. Zhang, X. Zhang, DVIO: An optimization-based tightly coupled direct visual-inertial odometry, IEEE Transactions  on Industrial Electronics,  online DOI: 10.1109/TIE.2020.3036243
7. Q. Sun, J. Yuan, X. Zhang, F. Duan, Plane-Edge-SLAM: seamless fusion of planes and edges for SLAM in indoor environments, IEEE Transactions on Automation Science and Engineering,  online  DOI: 10.1109/TASE.2020.3032831.
8. J. Yuan, W. Zhu, X. Dong, F. Sun, X. Zhang, Q. Sun, Y. Huang, A novel approach to image-sequence-based mobile robot place recognition, IEEE Transactions on Systems, Man, and Cybernetics: Systems, Online DOI: 10.1109/TSMC.2019.2956321.
9. J. Yuan, J. Cai, X. Zhang, Q. Sun, F. Sun, W. Zhu, Fusing skeleton recognition with face-TLD for human following of mobile service robots, IEEE Transactions  on Systems, Man and Cybernetics: Systems, Online DOI: 10.1109/TSMC.2019.2921974.
10. R. Wang, X. Zhang*, Y. Fang, Visual tracking of mobile robots with both velocity and acceleration saturation constraints, Mechanical Systems and Signal Processing,  2021,150: 107274.
11. F. Jiang, X. Zhang*, X. Chen, Y. Fang. Distributed optimization of visual sensor networks for coverage of a large-scale 3-D scene, IEEE/ASME Transactions on Mechatronics,   2020, 25(6):  2777-2788.
12. P. Shen, X. Zhang*, Y. Fang, and M. Yuan, Real-time acceleration-continuous path-constrained trajectory planning with built-in tradeoff between cruise and time-optimal motions, IEEE Transactions on Automation Science and Engineering, 2020, 17(4): 1911-1924.
13. X.T. Zhang, Y. Fang, X. Zhang, J. Jiang and X. Chen, Dynamic image-based output feedback control for visual servoing of multirotors, IEEE Transactions on Industrial Informatics,  2020, 16(12): 7624-7636.
14. X.T. Zhang, Y. Fang, X. Zhang, P. Shen, J. Jiang and X. Chen, Attitude-constrained time-optimal trajectory planning for rotorcrafts: theory and application to visual servoing, IEEE/ASME Transactions on Mechatronics, 2020, 25(4): 1912-1921.
15. X.T. Zhang, Y. Fang, X. Zhang, J. Jiang and X. Chen, A novel geometric hierarchical approach for dynamic visual servoing of quadrotors, IEEE Transactions on Industrial Electronics, 2020, 67(5): 3840-3849.
16. J. Wen, X. Zhang*, H. Gao, J. Yuan, Y. Fang, CAE-RLSM: consistent and efficient redundant line segment merging for online feature map building, IEEE Transactions on Instrumentation and Measurement, 2020, 69(7): 4222-4237.
17. M. Zhao, X. Guo*, X. Zhang,  Y. Fang, Y. Ou. ASPW-DRL: assembly sequence planning for workpieces via a deep reinforcement learning approacheces, Assembly Automation, 2019, vol. 40, no. 1, pp. 65-75.
18. H. Gao, X. Zhang*, J. Yuan, J. Song, Y. Fang, A novel global localization approach based on structural unit encoding and multiple hypothesis tracking, IEEE Transactions on Instrumentation and Measurement, 2019, 68(11): 4427-4442.
19. H. Gao, X. Zhang*, J. Wen, J. Yuan, Y. Fang, Autonomous indoor exploration via graph-based SLAM using directional endpoint features and polygon map construction, IEEE Transactions on Automation Science and Engineering, 2019, 16(4): 1531-1542.
20. X. Zhang, R. Wang, Y. Fang, B. Li, and B. Ma, Acceleration-level pseudo-dynamic visual servoing of mobile robots with backstepping and dynamic surface control, IEEE Transactions  on Systems, Man and Cybernetics: Systems, 2019, 49(10): 2071-2081.
21. X. Zhang, J. Wang, Y. Fang, H. Gao, J. Yuan, Multilevel humanlike motion planning for mobile robots in complex indoor environments, IEEE Transactions on Automation Science and Engineering, 2019, 16(3): 1244-1258.
22. B. Li, X. Zhang*, Y. Fang, W. Shi,  Visual servoing of wheeled mobile robots without desired images, IEEE Transactions on Cybernetics, 2019, 49(8): 2835-2844.
23. X. Zhang, X. Chen*, F. Farzadpour, and Y. Fang, A visual distance approach for multi-camera deployment with coverage optimization, IEEE/ASME Transactions on Mechatronics, 2018, 23(3): 1007-1018
24. P. Shen, X. Zhang*, and Y. Fang, Complete and time-optimal path-constrained trajectory planning with torque and velocity constraints: theory and applications, IEEE/ASME Transactions on Mechatronics, 2018, 23(2): 735-746.
25. B. Li, X. Zhang*, Y. Fang, W. Shi, Visual servo regulation of wheeled mobile robots with simultaneous depth identification, IEEE Transactions  on Industrial Electronics, 2018, 65(1): 460-469.
26. P. Shen, X. Zhang*, Y. Fang, Essential properties of numerical integration for time-optimal path-constrained trajectory planning, IEEE Robotics and Automation Letters, 2017, 2(2): 888-895.  arXiv:1610.02881.
27. X. Zhang, Y. Fang, B. Li, J. Wang, Visual servoing of nonholonomic mobile robots with uncalibrated camera-to-robot parameters, IEEE Transactions on Industrial Electronics, 2017, 64(1): 390-400.
28. X. Zhang, X. Chen, J. Alarcon, Y. Fang, 3-D model-based multi-camera deployment: a recursive convex optimization approach, IEEE/ASME Transactions on Mechatronics, 2015, 20(6): 3157-3169.
29. Z. Zhao, X. Zhang*, Y. Fang. Stacked multilayer self-organizing map for background modeling, IEEE Transactions on Image Processing, 2015, 24(9): 2841-2850.[Project Homepage][Source code]
30. X. Zhang, Y. Fang, N. Sun. Visual servoing of mobile robots for posture stabilization: from theory to experiments. International Journal of Robust and Nonlinear Control, 2015, 25(1): 1-15.
31. X. Zhang, Y. Fang, N. Sun. Minimum-time trajectory planning for underactuated overhead crane systems with state and control constraints. IEEE Transactions on Industrial Electronics, 2014, 61(12): 6915-6925.
32. X. Zhang, Y. Fang, X. Liu. Motion-estimation-based visual servoing of nonholonomic mobile robots. IEEE Transactions on Robotics (T-RO), 2011, 27(6): 1167-1175.
33. X. Zhang, Y. Fang, Y. Zhang. Discrete-time control of chained non-holonomic systems. IET Control Theory and Applications, 2011, 5(4): 640-646.
34. X. Zhang*, B. Zhang, X. Chen, Y. Fang. Coverage optimization of visual sensor networks: survey and comparison,  International Journal of Intelligent Robotics and Applications, Springer,  2019, 3(4): 342–361.
35. H. Gao, X. Zhang*, Y. Fang, J. Yuan, A line segment extraction algorithm using laser data based on seeded region growing, Int. Journal of Advanced Robotic Systems, 2018, 15(1): 1729881418755245.
36. X. Zhang, C. Wang, Y. Fang, H. Lu, X. Chen, Global homography calibration for monocular vision-based pose measurement of mobile robots, International Journal of Intelligent Robotics and Applications, Springer, 2017, 1(4): 372-382.
37. Y. Qiu, B. Li, W. Shi, X. Zhang, Visual Servo Tracking of Wheeled Mobile Robots with Unknown Extrinsic Parameters, IEEE Transactions on Industrial Electronics, 2019, 66(11), pp. 8600-8609.
38. Q. Sun, J. Yuan, X. Zhang, F. Sun, RGB-D SLAM in Indoor Environments with STING-Based Plane Feature Extraction, IEEE Transactions on Mechatronics, 2018, 23(3): 1071-1082.
39. Q. Lu, L. Yu, D. Zhang, X. Zhang. Simultaneous tracking and regulation visual servoing of wheeled mobile robots with uncalibrated extrinsic parameters. International Journal of Systems Science, 2018, 49(1): 217-229.
40. Ke, Z. Li, H. Xiao, X. Zhang. Visual servoing of constrained mobile robots based on model predictive control, IEEE Transactions on Systems, Man and Cybernetics: Systems, 2017, 47(7): 1428-1438.
41. H. Lu, Y. Fang, X. Ren, X. Zhang. Improved direct inverse tracking control of a piezoelectric tube scanner for high-speed AFM imaging, Mechatronics, 2015, 31: 189-195.
42. B. Li, Y. Fang, X. Zhang. Visual servo regulation of wheeled mobile robots with an uncalibrated onboard camera, IEEE/ASME Transactions on Mechatronics, 2015, 21(5): 2330-2342.
43. B. Li, Y. Fang, G. Hu, and X. Zhang, Model-free unified tracking and regulation visual servoing of wheeled mobile robots, IEEE Transactions on Control System Technology, 2016, 24(4): 1328-1339.
44. B. Li, Y. Fang, X. Zhang. Essential-matrix-based visual servoing of mobile robots without short baseline degeneration, International Journal of Robotics and Automation, 2015, 30(4).
45. N. Sun, Y. Fang, and X. Zhang, Energy coupling output feedback control of 4-DOF underactuated cranes with saturated inputs, Automatica, 2013, 49(5): 1318-1325.
46. N. Sun, Y. Fang, X. Zhang, and Y. Yuan, Transportation task-oriented trajectory planning for underactuated overhead cranes using geometric analysis, IET Control Theory and Applications, vol. 6, no. 10, pp. 1410-1423, Jul. 2012.
47. N. Sun, Y. Fang, and X. Zhang, An increased coupling-based control method for underactuated crane systems: Theoretical design and experimental implementation, Nonlinear Dynamics, vol. 70, no. 2, pp. 1135-1146, 2012.
48. Y. Fang, B. Ma, P. Wang, X. Zhang. A motion planning based adaptive control method for an overhead crane system. IEEE Transactions on Control Systems Technology (T-CST), 2012, 20(1): 241-248.
49. Y. Fang, X. Liu, X. Zhang. Adaptive active visual servoing of nonholonomic mobile robots. IEEE Transactions on Industrial Electronics (T-IE), 2012, 59(1): 486-497.

国内期刊(Domestic Journals)

1. 赵铭慧，张雪波*，郭宪，欧勇盛，基于深度强化学习的装配工序双向规划，控制理论与应用，2021, 已录用。
2. 赵铭慧，张雪波*，郭宪，欧勇盛. 基于分层强化学习的通用装配序列规划算法. 控制与决策，2020, 已录用。
3. 张一淳，张雪波*，方勇纯，沈佩尧. 基于万向轮的全方位移动机器人航位推算与跟踪控制研究. 机器人， 37, No. 3, pp. 361-368, 2015.
4. Y. Zhang, X. Zhang, Y. Fang. Dead Reckoning and Tracking Control of Omnidirectional Mobile Robots with Active Caster Wheels, Robot, 2015, 37(3): 361-368。
5. 张雪波, 路晗, 方勇纯，李宝全. 室外环境下PTZ摄像机全自动标定技术及其应用[J]. 机器人, 2013, 35(4): 385-393. (期刊网站首页图片新闻推荐阅读)。[实验录像]
6. X. Zhang, H. Lu, FANG Yongchun etc. An Automatic Calibration Method for Outdoor PTZ Cameras and Its Applications[J]. ROBOT, 2013, 35(4): 385-393.
7. 张雪波, 方勇纯, 刘玺. 移动机器人自适应视觉伺服镇定控制. 控制理论与应用, 2010, 27(9): 1123-1130.
8. 张雪波，方勇纯，马博军. 基于单应矩阵的摄像机标定方法及应用. 控制工程, 2010, 17(2): 248-251.
9. Zhang X, Fang Y, Ma B. A PFM-Based Global Convergence Visual Servo Path Planner. Acta Automatica Sinica (自动化学报), 2008, 34(10): 1250-1256.
10. 董星亮, 苑晶*, 黄枢子, 杨少坤,张雪波, 孙凤池, 黄亚楼. 室内环境下基于平面与线段特征的RGB-D视觉里程计, 机器人, 2018, 40 (6): 921-932.
11. 董星亮, 苑晶*, 张雪波, 黄亚楼. 室内环境下基于图像序列拓扑关系的移动机器人全局定位, 机器人, 2019, 41(1): 83-94.
12. 李宝全，方勇纯，张雪波. 基于2D三焦点张量的移动机器人视觉伺服镇定控制. 自动化学报，2014, 40(12): 2714-2723.
13. 辛哲奎, 方勇纯, 张雪波. 小型无人机地面跟踪系统机载云台自适应跟踪控制. 控制理论与应用, 2010, 27(8): 1001-1006.
14. 苑英海, 方勇纯, 张雪波. 基于全局代价函数优化的立体标定方法. 仪器仪表学报, 2011, 32(10): 2228-2234.

国际会议(International Conferences)

1. J. Wen, X. Zhang, Q. Bi, Z. Pan, Y. Feng, J. Yuan, Y. Fang, MRPB 1.0: A unified benchmark for the evaluation of mobile robot local planning approaches, Proc. of 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, accepted.
2. B. Zhang, X. Zhang, X. Chen and Y. Fang, “Grid Map Guided Indoor 3D Reconstruction for Mobile Robots with RGB-D Sensors,” 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Auckland, 2018, pp. 498-503.
3. P. Shen, X. Zhang*, Y. Fang, Essential properties of numerical integration for time-optimal trajectory planning along a specified path, IEEE Robotics and Automation Letters (RAL with ICRA option), 2017, accepted,  arXiv:1610.02881.
4. B. Zhang, X. Zhang*, X. Chen, and Y. Fang, A differential evolution approach for coverage optimization of visual sensor networks with parallel occlusion detection, The 2016 IEEE International Conference on Advanced Intelligent Mechatronics, 2016.07, Banff, pp. 1246-1251.
5. X. T. Zhang, Y. Fang, X. Liang, X. Zhang, Geometric adaptive dynamic visual servoing of a quadrotor UAV, Proc. of 2016 IEEE International Conference on Advanced Intelligent Mechatronics, Banff, Canada, 2016, pp. 312-317.
6. X. Zhang, R. Xue, Y. Yang, L. Cheng, Y. Fang, Learning time-optimal anti-swing trajectories for overhead crane systems, The 13th International Symposium on Neural Networks, 2016. 07, pp.338-345.
7. X. Zhang, X. Chen, X. Liang, Y. Fang. Distributed coverage optimization for deployment of directional sensor networks, IEEE Conference on Decision and Control, 2015, Osaka, Japan, pp. 246-251.
8. P. Shen, Y. Fang, X. Zhang, Trajectory planning of omnidirectional mobile robots with active casters: multi-motor coordination and singularity avoidance, Proc. of 5th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems, Shenyang, China, Jun. 2015, pp. 151-156.
9. X. Zhang, J. Alarcon, X. Chen*, Coverage enhancement for deployment of multi-camera networks, IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2015, Busan, South Korea, 909-914.
10. B. Li, Y. Fang, X. Zhang. Projection homography based uncalibrated visual servoing of wheeled mobile robots, Proc. of IEEE Conference on Decision and Control (CDC), Los Angeles, USA, 2014, 2167 – 2172.
11. X. Zhang, C. Wang, Y. Fang, K. Xing. Planar motion estimation from three-dimensional scenes, Proceedings of the 2014 IROS Workshop on Visual Control of Mobile Robots, Chicago, 2014, pp. 21-26.
12. Y. Huang, X. Zhang*, Y. Fang. Vision-based minimum-time planning of mobile robots with kinematic and visibility constraints. Proc. of the 19th IFAC World Congress, Cape Town, South Africa, 2014, pp. 11878-11883.
13. X. Zhang, J. Alarcon, X. Chen. Optimization for 3D Model-based Multi-Camera Deployment. Proc. of the 19th IFAC World Congress, Cape Town, South Africa, 2014, pp. 10127-10131.
14. X. Zhang, Y. Li, Y. Fang. Implicit feedback stabilizer-based explicit trajectory planning of mobile robots with kinematic constraints, Proc. of 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014, 4600-4605.
15. C. Chen, Y. He, C. Bu, J. Han, X. Zhang, Quartic Bezier curve based trajectory generation for autonomous vehicles with curvature and velocity constraints, Proc. of 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014, 6108-6113.
16. J. Alarcon-Herrera, X. Chen, X. Zhang, Viewpoint selection for vision systems in industrial inspection, Proc. of 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014, 4934-4939.
17. B. Li, Y. Fang, X. Zhang. Uncalibrated visual servoing of nonholonomic mobile robots, Proc. of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, 2013, 584-589.
18. W. He, Y. Fang, X. Zhang. Prediction-based Interception Control Strategy Design with a Specified Approach Angle Constraint for Wheeled Service Robots, Proc. of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, 2013, 2725-2730.
19. X. Zhang, Y. Fang. Set-oriented optimal path planning of mobile robots by a polynomial rootfinder. Proc. of 2013 IEEE Multi-conference on Systems and Control (CCA), Hyderabad, India, 2013.08, pp. 778-783.
20. Z. Zhao, T. Bouwmans, X. Zhang*, Y. Fang. A fuzzy background modeling approach for motion detection in dynamic backgrounds. International Conference on Multimedia and Signal Processing (CMSP2012), pp. 177-185, 2012. (Acceptance ratio: 24.08%)
21. X. Zhang, Y. Fang, X. Liu. Visual servoing of nonholonomic mobile robots based on a new motion estimation technique. Proceedings of IEEE Conference on Decision and Control (CDC), Shanghai, 2009, 8428-8433.