使用udpscr时，Gstreamer插件无法在Android上正确播放视频

我在Android上实现gstreamer插件播放RTP视频时遇到了一些问题。我有以下代码（工作正常）：

而对另一个管道使用相同的代码（基于udpsrc而不是playbin3）则不行。我在本例中使用的管道是：

udpsrc端口=53512！应用程序/x-rtp，媒体=视频，时钟频率=90000，编码名称=H264，有效负载=96！rtph264depay！解码BIN3！笨蛋！glcolorconvert！视频/x-raw（内存：GLMemory），格式=RGBA，纹理目标=2D！fakesink sync=0 qos=1 name=sink

代码如下：

      full_pipeline_description = g_strdup_printf("%s", pipeline_cmd);
  gub_log_pipeline(pipeline, "Using pipeline: %s", full_pipeline_description);
  pipeline->pipeline = gst_parse_launch(full_pipeline_description, &err);
  g_free(full_pipeline_description);
  if (err) {
    gub_log_pipeline(pipeline, "Failed to create pipeline: %s", err->message);
    return;
  }

  vsink = gst_parse_bin_from_description(gub_get_video_branch_description(), TRUE, NULL);
  gub_log_pipeline(pipeline, "Using video sink: %s", gub_get_video_branch_description());
  g_object_set(pipeline->pipeline, "sink", vsink, NULL);
  g_object_set(pipeline->pipeline, "flags", 0x0003, NULL);

  bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline->pipeline));
  gst_bus_add_signal_watch(bus);
  gst_object_unref(bus);
  g_signal_connect(bus, "message", G_CALLBACK(message_received), pipeline);

  // Plant a pad probe to answer context queries
  GstElement *sink;
  sink = gst_bin_get_by_name(GST_BIN(vsink), "sink");
  if (sink) {
    GstPad *pad = gst_element_get_static_pad(sink, "sink");
    if (pad) {
      gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM, pad_probe, pipeline, NULL);
      gst_object_unref(pad);
    }
    gst_object_unref(sink);
  }

ffmpeg -f lavfi -i testsrc -vf scale=1280:960 -vcodec libx264 -profile:v baseline -pix_fmt yuv420p -f rtp rtp://YOUR_IP:PORT

基本上，在这种情况下，我只看到空白窗口（不同的颜色）。执行过程中唯一的区别是，在使用playbin3时调用pad\u探针，而在使用udpsrc时不调用。这是我能看到的添加一些日志的唯一区别。我想了解为什么在使用udpsrc时不调用此回调，以及我是否丢失了某些内容或使用了错误的内容

我在使用gstreamer-1.14.4和1.16.2版本时面临相同的问题。任何提示都非常受欢迎。

g\u对象集（管道->管道，“接收器”，vsink，NULL）实际上什么都不做；GstPipeline没有“sink”属性（与playbin不同）。通常情况下，它会发出一个日志警告，确切地说明这一点

要将接收器添加到管道中，您需要像在GStreamer应用程序中通常那样执行此操作：您找到需要连接的源焊盘，或者等待正确的源焊盘链接它，以在“添加焊盘”信号中显示（例如，在
decodebin
中发生这种情况）.
最后，经过一些调查，并基于此线程，我找到了问题的根本原因。基本上，正如我所怀疑的那样，在使用udpsrc时没有调用pad probe的回调，它只在使用playbin3时起作用。因此，没有提供图形上下文，视频也没有正确复制。为了解决这个问题，我必须添加逻辑来处理总线上的消息，以正确回答GST\u MESSAGE\u NEED\u上下文请求。为此，首先必须连接回调以处理总线消息，如下所示：

      full_pipeline_description = g_strdup_printf("%s", pipeline_cmd);
  gub_log_pipeline(pipeline, "Using pipeline: %s", full_pipeline_description);
  pipeline->pipeline = gst_parse_launch(full_pipeline_description, &err);
  g_free(full_pipeline_description);
  if (err) {
    gub_log_pipeline(pipeline, "Failed to create pipeline: %s", err->message);
    return;
  }

  vsink = gst_parse_bin_from_description(gub_get_video_branch_description(), TRUE, NULL);
  gub_log_pipeline(pipeline, "Using video sink: %s", gub_get_video_branch_description());
  g_object_set(pipeline->pipeline, "sink", vsink, NULL);
  g_object_set(pipeline->pipeline, "flags", 0x0003, NULL);

  bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline->pipeline));
  gst_bus_add_signal_watch(bus);
  gst_object_unref(bus);
  g_signal_connect(bus, "message", G_CALLBACK(message_received), pipeline);

  // Plant a pad probe to answer context queries
  GstElement *sink;
  sink = gst_bin_get_by_name(GST_BIN(vsink), "sink");
  if (sink) {
    GstPad *pad = gst_element_get_static_pad(sink, "sink");
    if (pad) {
      gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM, pad_probe, pipeline, NULL);
      gst_object_unref(pad);
    }
    gst_object_unref(sink);
  }

ffmpeg -f lavfi -i testsrc -vf scale=1280:960 -vcodec libx264 -profile:v baseline -pix_fmt yuv420p -f rtp rtp://YOUR_IP:PORT

g_信号连接（总线，“消息”，g_回调（消息接收），管道）

然后在message_received函数中，我添加了以下代码

static void message_received(GstBus *bus, GstMessage *message, GUBPipeline *pipeline) {
    switch (GST_MESSAGE_TYPE(message)) {
    ...
      case GST_MESSAGE_NEED_CONTEXT:
    {
        const gchar *context_type;
        GstContext *context = NULL;
        gst_message_parse_context_type (message, &context_type);
        context = gub_provide_graphic_context(pipeline->graphic_context, context_type);
         if (context)
         {
             gst_element_set_context (GST_ELEMENT (message->src), context);
             gst_context_unref (context);
         }
        break;
    }
    ...
}

通过这些修改，我现在能够正确地接收和复制视频。RTP视频流使用ffmpeg testsrc工具进行模拟，如下所示：

      full_pipeline_description = g_strdup_printf("%s", pipeline_cmd);
  gub_log_pipeline(pipeline, "Using pipeline: %s", full_pipeline_description);
  pipeline->pipeline = gst_parse_launch(full_pipeline_description, &err);
  g_free(full_pipeline_description);
  if (err) {
    gub_log_pipeline(pipeline, "Failed to create pipeline: %s", err->message);
    return;
  }

  vsink = gst_parse_bin_from_description(gub_get_video_branch_description(), TRUE, NULL);
  gub_log_pipeline(pipeline, "Using video sink: %s", gub_get_video_branch_description());
  g_object_set(pipeline->pipeline, "sink", vsink, NULL);
  g_object_set(pipeline->pipeline, "flags", 0x0003, NULL);

  bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline->pipeline));
  gst_bus_add_signal_watch(bus);
  gst_object_unref(bus);
  g_signal_connect(bus, "message", G_CALLBACK(message_received), pipeline);

  // Plant a pad probe to answer context queries
  GstElement *sink;
  sink = gst_bin_get_by_name(GST_BIN(vsink), "sink");
  if (sink) {
    GstPad *pad = gst_element_get_static_pad(sink, "sink");
    if (pad) {
      gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM, pad_probe, pipeline, NULL);
      gst_object_unref(pad);
    }
    gst_object_unref(sink);
  }

ffmpeg -f lavfi -i testsrc -vf scale=1280:960 -vcodec libx264 -profile:v baseline -pix_fmt yuv420p -f rtp rtp://YOUR_IP:PORT