Python Appengine:同一调用的Memcache在5秒内丢失??

Python Appengine:同一调用的Memcache在5秒内丢失??,python,google-app-engine,caching,memcached,Python,Google App Engine,Caching,Memcached,这是我的memcache获取和设置代码(所有内容都指向_mc_get) 此功能在appengine上花费的时间超过40秒 @app.route('/orders/schedule/all/', methods=['GET', 'POST']) @login_required def generate_schedule_for_all_orders(): """Merge all the orders together in one big schedule TODO: put rang

这是我的memcache获取和设置代码(所有内容都指向_mc_get)

此功能在appengine上花费的时间超过40秒

@app.route('/orders/schedule/all/', methods=['GET', 'POST'])
@login_required
def generate_schedule_for_all_orders():
  """Merge all the orders together in one big schedule

  TODO: put range by default.
  TODO: think of default empty scenario
  """
  orders = key_dict(mc_get_all_orders())

  # TODO: somehow there's a lot of orphaned tasks in development. For now,
  # we will just query for non orphaned tasks to prevent errors piling up
  tasks = mc_get_tasks_by_orders(orders.keys())

  ds = sorted([t.date for t in tasks])
  start, end = ds[0], ds[-1]
  dates = range_for_dates(start, end)

  return render_template('backward_schedule.html',
                         tasks=tasks,
                         orders=orders,
                         producetypes=key_dict(mc_get_all_producetypes()),
                         dates=dates,
                         display_order_info=True)
我想知道为什么花了这么长时间,所以我在我的_mc_上插入了logging.debug语句,以查看是否命中或丢失了memcache。令我惊讶的是,如果我等待的时间超过3秒,memcache上的数据将丢失100%。(这适用于我的开发人员和生产服务器)

首次访问页面的日志:记下时间

DEBUG    2014-06-11 23:16:11,765 mc.py:27] Memcache: missing + adding Order:all
DEBUG    2014-06-11 23:16:11,808 mc.py:27] Memcache: missing + adding Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
DEBUG    2014-06-11 23:16:12,255 mc.py:27] Memcache: missing + adding ProduceType:all
INFO     2014-06-11 23:16:13,517 recording.py:665] Saved; key: __appstats__:071700, part: 105 bytes, full: 64362 bytes, overhead: 0.002 + 0.010; link: http://localhost:8080/_ah/stats/details?time=1402528571756
第二次访问页面(第一次访问完成后立即重新加载)

第三次访问同一页面(只需点击刷新):(这实际上是第二次访问同一页面后4秒)

为什么我的memcache会这么快过期

  • memcache条目没有保证,理论上,由于内存压力,它可以随时被逐出

  • 您可以使用memcache.add,它仅在不存在时设置值。您的超时时间为10秒,memcache是在23:16:13添加的,当您在23:16:23访问它时,它将丢失。这是预期的,因为它大约超过10秒。它从不打印memcache add失败,因为如果找到该值,函数将立即返回

  • 有点奇怪的是,mc_获取所有订单和mc_获取任务都使用相同的键,但查询不同

  • memcache条目没有保证,理论上,由于内存压力,它可以随时被逐出

  • 您可以使用memcache.add,它仅在不存在时设置值。您的超时时间为10秒,memcache是在23:16:13添加的,当您在23:16:23访问它时,它将丢失。这是预期的,因为它大约超过10秒。它从不打印memcache add失败,因为如果找到该值,函数将立即返回

  • 有点奇怪的是,mc_获取所有订单和mc_获取任务都使用相同的键,但查询不同


  • 您正在将过期时间设置为10秒。客户机和服务器之间可能存在时间偏差。将其设置为600进行测试,看看情况是否好转。您将过期时间设置为10秒。客户机和服务器之间可能存在时间偏差。将其设置为600进行测试,看看情况是否好转。天哪,我真不敢相信我错过了。我认为10是添加到memcache所需的时间,如果10秒内没有完成,它将发出故障警报。谢谢天哪,我真不敢相信我错过了。我认为10是添加到memcache所需的时间,如果10秒内没有完成,它将发出故障警报。谢谢
    DEBUG    2014-06-11 23:16:11,765 mc.py:27] Memcache: missing + adding Order:all
    DEBUG    2014-06-11 23:16:11,808 mc.py:27] Memcache: missing + adding Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
    DEBUG    2014-06-11 23:16:12,255 mc.py:27] Memcache: missing + adding ProduceType:all
    INFO     2014-06-11 23:16:13,517 recording.py:665] Saved; key: __appstats__:071700, part: 105 bytes, full: 64362 bytes, overhead: 0.002 + 0.010; link: http://localhost:8080/_ah/stats/details?time=1402528571756
    
    INFO     2014-06-11 23:16:13,534 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559
    DEBUG    2014-06-11 23:16:15,638 mc.py:24] Memcache: hit Order:all
    DEBUG    2014-06-11 23:16:15,699 mc.py:24] Memcache: hit Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
    DEBUG    2014-06-11 23:16:15,706 mc.py:24] Memcache: hit ProduceType:all
    INFO     2014-06-11 23:16:16,937 recording.py:665] Saved; key: __appstats__:075600, part: 56 bytes, full: 15643 bytes, overhead: 0.000 + 0.004; link: http://localhost:8080/_ah/stats/details?time=1402528575619
    
    INFO     2014-06-11 23:16:17,112 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559
    DEBUG    2014-06-11 23:16:23,362 mc.py:27] Memcache: missing + adding Order:all
    DEBUG    2014-06-11 23:16:23,415 mc.py:27] Memcache: missing + adding Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
    DEBUG    2014-06-11 23:16:23,888 mc.py:27] Memcache: missing + adding ProduceType:all
    INFO     2014-06-11 23:16:25,153 recording.py:665] Saved; key: __appstats__:083300, part: 105 bytes, full: 64339 bytes, overhead: 0.002 + 0.011; link: http://localhost:8080/_ah/stats/details?time=1402528583353
    INFO     2014-06-11 23:16:25,169 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559