Python 优化社交排行榜
我正在使用Google App Engine（python）作为移动社交游戏的后端。该游戏使用Twitter集成，允许人们关注相关排行榜，并与他们的朋友或追随者比赛 到目前为止，这个难题中最昂贵的部分是后台（push）任务，它点击twitterapi查询给定用户的朋友和追随者，然后将这些数据存储在我们的数据存储中。我正试图优化它，尽可能降低成本 数据模型： 与此部分应用程序相关的主要模型有三种：Python 优化社交排行榜,python,google-app-engine,optimization,twitter,leaderboard,Python,Google App Engine,Optimization,Twitter,Leaderboard,我正在使用Google App Engine（python）作为移动社交游戏的后端。该游戏使用Twitter集成，允许人们关注相关排行榜，并与他们的朋友或追随者比赛 到目前为止，这个难题中最昂贵的部分是后台（push）任务，它点击twitterapi查询给定用户的朋友和追随者，然后将这些数据存储在我们的数据存储中。我正试图优化它，尽可能降低成本 数据模型： 与此部分应用程序相关的主要模型有三种： User '''General user info, like scores and stats''
User
'''General user info, like scores and stats'''
# key id => randomly generated string that uniquely identifies a user
# along the lines of user_kdsgj326
# (I realize I probably should have just used the integer ID that GAE
# creates, but its too late for that)
AuthAccount
'''Authentication mechanism.
A user may have multiple auth accounts- one for each provider'''
# key id => concatenation of the auth provider and the auth provider's unique
# ID for that user, ie, "tw:555555", where '555555' is their twitter ID
auth_id = ndb.StringProperty(indexed=True) # ie, '555555'
user = ndb.KeyProperty(kind=User, indexed=True)
extra_data = ndb.JsonProperty(indexed=False) # twitter picture url, name, etc.
RelativeUserScore
'''Denormalization for quickly generated relative leaderboards'''
# key id => same as their User id, ie, user_kdsgj326, so that we can quickly
# retrieve the object for each user
follower_ids = ndb.StringProperty(indexed=True, repeated=True)
# misc properties for the user's score, name, etc. needed for leaderboard
twitter_friend_ids = twitter_api.friend_ids() # potentially 5000 values
friend_system_ids = AuthAccount\
.query(AuthAccount.auth_id.IN(twitter_friend_ids))\
.fetch(projection=[AuthAccount.user_id])
我会这样组织任务：
- 从memcache请求数据，如果数据不存在，则调用
fetch\u async（）
future = twitter_api.friend_ids() # make this asynchronous
auth_users = memcache.get('auth_users')
if auth_users is None:
auth_accounts = AuthAccount.query()
.fetch(projection=[AuthAccount.auth_id,
AuthAccount.user_id])
auth_users = dict([(a.auth_id, a.user_id) for a in auth_accounts])
memcache.add('auth_users', auth_users, 60)
twitter_friend_ids = future.get_result() # get async twitter results
friend_system_ids = []
for id in twitter_friend_ids:
friend_id = auth_users.get("tw:%s" % id)
if friend_id:
friend_system_ids.append(friend_id)
twitter_friend_ids = twitter_api.friend_ids() # potentially 5000 values
auth_account_keys = [ndb.Key("AuthAccount", "tw:%s" % id) for id in twitter_friend_ids]
friend_system_ids = filter(None, ndb.get_multi(auth_account_keys))
这将使用ndb的内置memcache在使用带有键的
get_multi（）tw_auth_accounts=ndb.get_multi（[ndb.Key（authcount），tw:%s”%id）用于tw_id中的id]）map（）Nonefilter（）