Elixir 长生不老药；EXTO：视图中的子查询或侧面加载和规范化json输出

我最近开始将现有的PHP messenger应用程序移植到elixir（使用elixir 1.3、phoenix 1.2、Exto 2.0.1和mariaex 0.7.7）。该应用服务于数百万用户，因此性能非常重要。我对长生不老药很陌生，所以请原谅我这个问题的愚蠢

我有以下数据库架构：

每个线程都有多个线程参与者和消息。thread_参与者在链接线程的上下文中拥有关于用户的信息（例如，当用户最后一次看到此线程时）。线程由用户编写的多条消息组成

我希望json my API最终返回的是：

"data": {
  "result": [1, 2],
  "threads": {
    1: {
      "id": 1,
      "unread_count": 2,
      "starred": false,
      "muted": false,
      "last_seen": "2015-10-20T19:01:46",
      "participants": [1, 2]
    },
    22: {
      "id": 22,
      "unread_count": 0,
      "starred": true,
      "muted": false,
      "last_seen": "2016-06-20T12:00:00",
      "participants": [1, 3]
    }
  },
  users: {
    1: {
      id: 1,
      name: 'John'
    },
    2: {
      id: 2,
      name: 'Dan'
    },
    3: {
      id: 3,
      name: 'Eric'
    }        
  }

以下是我的Thread和ThreadParticipant模式：

schema "thread" do
  field :created, Ecto.DateTime, usec: true, autogenerate: true
  belongs_to :creator, UserAbstract
  has_many :messages, ThreadMessage
  has_many :participants, ThreadParticipant
  has_many :users, through: [:participants, :user]
  field :last_seen, Ecto.DateTime, virtual: true, default: :null
  field :muted, :boolean, virtual: true, default: false
  field :starred, :boolean, virtual: true, default: false
  field :unread_count, :integer, virtual: true, default: 0
end

@primary_key false
schema "thread_participant" do
  belongs_to :thread, Messenger.Thread, primary_key: true
  belongs_to :user, Messenger.UserAbstract, primary_key: true
  field :last_seen, Ecto.DateTime, usec: true, autogenerate: true
  field :starred, :boolean, default: false
  field :muted, :boolean, default: false
end

我使用查询组合为用户上下文化线程列表：

def for_user(query, user_id) do
  from t in query,
  join: p in assoc(t, :participants),
  join: message in assoc(t, :messages),
  left_join: messageNew in ThreadMessage, on: messageNew.id == message.id and messageNew.created > p.last_seen,
  where: p.user_id == ^user_id,
  order_by: [desc: max(message.created)],
  group_by: t.id,
  select: %{ t | last_seen: p.last_seen, muted: p.muted,starred: p.starred, unread_count: count(messageNew.id)}
end

所以当我这么做的时候

 Thread |> Thread.for_user(user_id) |> Repo.all

我能够获得几乎所有正确的聚合信息，但由于group_by thread.id，我错过了参与者id

在纯SQL中，我会执行下面的代码，然后在代码中重建模型：

SELECT s.id, s.last_seen, s.muted, s.starred, s.last_message_date, s.unread_count, p.user_id
  FROM (
    SELECT t0.`id` , t2.`last_seen` , t2.`muted` , t2.`starred` , max(t1.`created`) as last_message_date, count(t3.id) as unread_count
    FROM `thread` AS t0
    INNER JOIN `thread_message` AS t1 ON t0.`id` = t1.`thread_id`
    INNER JOIN `thread_participant` AS t2 ON ( t0.`id` = t2.`thread_id` ) AND ( t2.`user_id` = 9854 )
    LEFT JOIN `thread_message` AS t3 ON t3.`id` = t1.`id` AND t3.`created` > t2.`last_seen`
    GROUP BY t0.`id`
  ) as s
  INNER JOIN `thread_participant` AS p ON p.`thread_id` = s.`id`
  ORDER BY s.`last_message_date` DESC

我所有将此转换为EXTO的尝试（即使使用子查询或片段）都失败了（子查询中没有Max（），子查询中的字段别名没有保留，…）

因此，除了第一个查询（for_user（）），我还要在第二个查询中加载参与者：

thread_ids = Enum.map(threads, fn (x) -> x.id end)

def get_participating_user(thread_ids) do
  from tp in ThreadParticipant,
  join: user in assoc(tp, :user),
  where: tp.thread_id in ^thread_ids,
  preload: :user
end

participants = Thread.get_participating_user(thread_ids) |> Repo.all

但现在我陷入了如何合并两个结果集的困境（将第二个查询中的ThreadParticipants放在第一个查询中的每个线程的参与者键下），然后如何在我的视图中正常化地输出它（只有参与者ID保存在thread.participants下，所有不同的用户都在users下输出）

---

在这个问题上我已经坚持了好几个小时了，我真的很感激你能分享的任何知识

我终于把所有的东西都做好了。在花了好几个小时重新设计轮子之后（即在第二个查询中加载线程参与者，然后查看线程列表以添加他们的参与者），我注意到无论您在第一个查询中输入什么，ecto都将在单独的查询中获取预加载的关联

因此，要解决问题1（如何合并两个结果集），解决方案是：不要这样做：-）只需将所需的关联标记为预加载。只要您在主查询中加载了线程ID，ecto就会很乐意为您完成这项艰巨的工作：

  def for_user(query, user_id) do
    from t in query,
    join: p in assoc(t, :participants),
    join: message in assoc(t, :messages),
    join: u in assoc(p, :user),
    left_join: messageNew in ThreadMessage, on: messageNew.id == message.id and messageNew.created > p.last_seen,
    where: p.user_id == ^user_id,
    order_by: [desc: max(message.created)],
    group_by: t.id,
    preload: [:participants,:users],
    select: %{ t | last_seen: p.last_seen, muted: p.muted,starred: p.starred, unread_count: count(messageNew.id)}
  end

在调试模式下，您可以看到EXTO执行以下查询：

SELECT t0.`id`, t0.`created`, t0.`creator_id`, t1.`last_seen`, t1.`muted`, t1.`starred`, count(t4.`id`) FROM `thread` AS t0 INNER JOIN `thread_participant` AS t1 ON t1.`thread_id` = t0.`id` INNER JOIN `thread_message` AS t2 ON t2.`thread_id` = t0.`id` INNER JOIN `user` AS u3 ON u3.`id` = t1.`user_id` LEFT OUTER JOIN `thread_message` AS t4 ON (t4.`id` = t2.`id`) AND (t4.`created` > t1.`last_seen`) WHERE (t1.`user_id` = ?) GROUP BY t0.`id` ORDER BY max(t2.`created`) DESC LIMIT 5 [20]

SELECT t0.`thread_id`, t0.`user_id`, t0.`last_seen`, t0.`starred`, t0.`muted`, t0.`thread_id` FROM `thread_participant` AS t0 WHERE (t0.`thread_id` IN (?,?,?,?,?)) ORDER BY t0.`thread_id` [45, 47, 66, 77, 88]

SELECT u0.`id`, u0.`display_name`, u0.`id` FROM `user` AS u0 WHERE (u0.`id` IN (?,?,?,?,?,?)) [10, 11, 12, 13, 14, 15] 

要解决问题2（在我看来，如何输出它，标准化（只有参与者ID保存在thread.participants下，所有不同的用户都在users下输出）），一旦您开始理解elixir的映射、列表和枚举，这就非常简单了：

控制器将线程列表传递给具有以下代码的视图：

def render("index.json", %{thread: threads}) do
%{ data:
  %{
    threads: render_many(threads, Messenger.ThreadView, "user_thread.json"),
    users: render_many(threads |> Stream.flat_map(&(&1.users)) |> Stream.uniq, Messenger.UserAbstractView, "user_abstract.json")
  }
}

def render("user_thread.json", %{thread: thread}) do
  %{id: thread.id,
    last_seen: thread.last_seen,
    muted: thread.muted,
    starred: thread.starred,
    unread_count: thread.unread_count,
    participants: Enum.map(thread.participants, fn(tp) -> tp.user_id end)
    }
end

棘手的部分：

#Here we extract a list of uniq users from our list of threads  
#and use our user view to render them
users: render_many(threads |> Stream.flat_map(&(&1.users)) |> Stream.uniq, Messenger.UserAbstractView, "user_abstract.json")

#Here we populate the participants key with a list of the participants ids
participants: Enum.map(thread.participants, fn(tp) -> tp.user_id end)

你就这样走了！->规范化结构

希望它能为你节省一些时间，如果你像我一样，用灵丹妙药这种美妙的语言来表达你的想法