Python多进程任务处理与返回值获取详解

Ivy

发布时间：2026-10-13| 6人浏览过

原创

本文介绍Python中使用multiprocessing.Pool进行多任务处理的两种方法。第一种通过apply_async提交任务并遍历结果对象获取返回值；第二种展示在异步提交后直接对特定任务调用get()获取结果。代码包含关键词提取与文本分类示例，演示如何并行执行并整合返回数据。

以下直接展示核心实现代码。

# coding:utf-8
from multiprocessing import Pool
import time
 
 
def keywords(title, content, top_n=5):
 print u'关键词提取...'
 print title, content, top_n
 time.sleep(3)
 return 0, [u"晴", u"多云"]
 
 
def category(title, content):
 print u'文本分类...'
 print title, content
 time.sleep(5)
 return 1, [u"天气"]
 
 
def run(title, content):
 keywords_list = []
 category_list = []
 pool = Pool(processes=2)
 q = []
 q.append(pool.apply_async(keywords, args=(title, content, 5)))
 q.append(pool.apply_async(category, args=(title, content)))
 for item in q:
  r = item.get()
  if r[0] == 0:
   keywords_list = r[1]
  elif r[0] == 1:
   category_list = r[1]
 pool.close()
 pool.join()
 
 return category_list, keywords_list
 
if __name__ == "__main__":
 title = u"天气预报"
 content = u"北京今日天气：晴转多云"
 t1 = time.time()
 category_list, keywords_list = run(title, content)
 print "分类结果：", " ".join(category_list)
 print "关键词提取结果", " ".join(keywords_list)
 print time.time() - t1

另一种实现方式如下：

# coding:utf-8
from multiprocessing import Pool
import time
 
 
def keywords(title, content, top_n=5):
 print u'关键词提取...'
 print title, content, top_n
 time.sleep(3)
 return 0, [u"晴", u"多云"]
 
 
def category(title, content):
 print u'文本分类...'
 print title, content
 time.sleep(5)
 return 1, [u"天气"]
 
 
def run(title, content):
 keywords_list = []
 category_list = []
 pool = Pool(processes=2)
 q = []
 q.append(pool.apply_async(keywords, args=(title, content, 5)))
 keywords_list = [w["word"] for w in q[0].get()[1]]
 category_list = category(title, content)[1]
 pool.close()
 pool.join()
 
 return category_list, keywords_list
 
if __name__ == "__main__":
 title = u"天气预报"
 content = u"北京今日天气：晴转多云"
 t1 = time.time()
 category_list, keywords_list = run(title, content)
 print "分类结果：", " ".join(category_list)
 print "关键词提取结果", " ".join(keywords_list)
 print time.time() - t1

上述关于Python多任务处理及返回值获取的方法即为本文分享的全部内容，希望能为读者提供有效参考，并欢迎大家继续支持本站。