Skip to content

The GenericFeed parsing problem #236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
followthemoney1 opened this issue Nov 9, 2022 · 3 comments
Open

The GenericFeed parsing problem #236

followthemoney1 opened this issue Nov 9, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@followthemoney1
Copy link

Is your feature request related to a problem? Please describe.
The problem is that we receiving the type of model from the stream as this:
image

When we tried to use FlatFeed it's says that there was a problem with parsing data like this:
image

So then we changed all FlatFeed using to the GenericFeed(want to say that documentation also should be improved for this), so after we changed the implementation, we started to receive the parsing data error of field 'foreign_id'
image


Such as I've understood and you can see from the response structure, we have a root called foreign_id and which also has a field inside with the same name, foreign_id
Untitled

so when the library tries to get this data from json response and extract the main keys from json, you can follow here:
image
it's said that foreign_id is not a type of String.

Describe the solution you'd like
From our side, we tried to just replace the data to another key and everything start to worked:
image

JSON Response from stream

{
  "results": [
    {
      "actor": {
        "created_at": "2022-11-04T10:34:26.427440Z",
        "updated_at": "2022-11-04T10:34:26.427440Z",
        "id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
        "data": {
          "first_name": "il",
          "last_name": "io"
        }
      },
      "foreign_id": {
        "id": "2c9200d9-1d07-4875-96c1-315353817809",
        "collection": "post",
        "foreign_id": "post:2c9200d9-1d07-4875-96c1-315353817809",
        "data": {
          "body": "ffff",
          "date": "2022-11-07T16:49:25.807000"
        },
        "created_at": "2022-11-07T16:49:26.4748Z",
        "updated_at": "2022-11-07T16:49:26.4748Z"
      },
      "id": "23563bf0-5ebc-11ed-8080-80012ce6320f",
      "object": {
        "id": "2c9200d9-1d07-4875-96c1-315353817809",
        "collection": "post",
        "foreign_id": "post:2c9200d9-1d07-4875-96c1-315353817809",
        "data": {
          "body": "ffff",
          "date": "2022-11-07T16:49:25.807000"
        },
        "created_at": "2022-11-07T16:49:26.4748Z",
        "updated_at": "2022-11-07T16:49:26.4748Z"
      },
      "origin": null,
      "target": "",
      "time": "2022-11-07T16:49:25.807000",
      "user_id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
      "verb": "post"
    },
    {
      "actor": {
        "created_at": "2022-11-04T10:34:26.427440Z",
        "updated_at": "2022-11-04T10:34:26.427440Z",
        "id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
        "data": {
          "first_name": "il",
          "last_name": "io"
        }
      },
      "foreign_id": {
        "id": "7c22a074-fba9-45a8-a6a3-734d7daa82bf",
        "collection": "post",
        "foreign_id": "post:7c22a074-fba9-45a8-a6a3-734d7daa82bf",
        "data": {
          "body": "Hhh",
          "date": "2022-11-07T11:07:51.037867"
        },
        "created_at": "2022-11-07T11:07:52.509356Z",
        "updated_at": "2022-11-07T11:07:52.509356Z"
      },
      "id": "6b80a4ae-5e8c-11ed-8080-8000357a1441",
      "object": {
        "id": "7c22a074-fba9-45a8-a6a3-734d7daa82bf",
        "collection": "post",
        "foreign_id": "post:7c22a074-fba9-45a8-a6a3-734d7daa82bf",
        "data": {
          "body": "Hhh",
          "date": "2022-11-07T11:07:51.037867"
        },
        "created_at": "2022-11-07T11:07:52.509356Z",
        "updated_at": "2022-11-07T11:07:52.509356Z"
      },
      "origin": null,
      "target": "",
      "time": "2022-11-07T11:07:51.037867",
      "user_id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
      "verb": "post"
    },
    {
      "actor": {
        "created_at": "2022-11-04T10:34:26.427440Z",
        "updated_at": "2022-11-04T10:34:26.427440Z",
        "id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
        "data": {
          "first_name": "il",
          "last_name": "io"
        }
      },
      "foreign_id": {
        "id": "d5bf20c2-b5be-41af-a6fe-10b8eed7ae88",
        "collection": "post",
        "foreign_id": "post:d5bf20c2-b5be-41af-a6fe-10b8eed7ae88",
        "data": {
          "body": "Yyg",
          "date": "2022-11-07T11:00:20.356377"
        },
        "created_at": "2022-11-07T11:00:21.43526Z",
        "updated_at": "2022-11-07T11:00:21.43526Z"
      },
      "id": "5ee01afa-5e8b-11ed-8080-80001eed094b",
      "object": {
        "id": "d5bf20c2-b5be-41af-a6fe-10b8eed7ae88",
        "collection": "post",
        "foreign_id": "post:d5bf20c2-b5be-41af-a6fe-10b8eed7ae88",
        "data": {
          "body": "Yyg",
          "date": "2022-11-07T11:00:20.356377"
        },
        "created_at": "2022-11-07T11:00:21.43526Z",
        "updated_at": "2022-11-07T11:00:21.43526Z"
      },
      "origin": null,
      "target": "",
      "time": "2022-11-07T11:00:20.356377",
      "user_id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
      "verb": "post"
    },
    {
      "actor": {
        "created_at": "2022-11-04T10:34:26.427440Z",
        "updated_at": "2022-11-04T10:34:26.427440Z",
        "id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
        "data": {
          "first_name": "il",
          "last_name": "io"
        }
      },
      "foreign_id": {
        "id": "c6a550a4-a7df-4119-87fc-56ecbf7b23eb",
        "collection": "post",
        "foreign_id": "post:c6a550a4-a7df-4119-87fc-56ecbf7b23eb",
        "data": {
          "body": "Vvv",
          "date": "2022-11-07T09:56:24.539436"
        },
        "created_at": "2022-11-07T09:56:25.855634Z",
        "updated_at": "2022-11-07T09:56:25.855634Z"
      },
      "id": "708ce3b8-5e82-11ed-8080-80003da76b62",
      "object": {
        "id": "c6a550a4-a7df-4119-87fc-56ecbf7b23eb",
        "collection": "post",
        "foreign_id": "post:c6a550a4-a7df-4119-87fc-56ecbf7b23eb",
        "data": {
          "body": "Vvv",
          "date": "2022-11-07T09:56:24.539436"
        },
        "created_at": "2022-11-07T09:56:25.855634Z",
        "updated_at": "2022-11-07T09:56:25.855634Z"
      },
      "origin": null,
      "target": "",
      "time": "2022-11-07T09:56:24.539436",
      "user_id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
      "verb": "post"
    },
    {
      "actor": {
        "created_at": "2022-11-04T10:34:26.427440Z",
        "updated_at": "2022-11-04T10:34:26.427440Z",
        "id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
        "data": {
          "first_name": "il",
          "last_name": "io"
        }
      },
      "foreign_id": {
        "id": "e147b773-d13d-4952-ba14-c0faec1b6794",
        "collection": "post",
        "foreign_id": "post:e147b773-d13d-4952-ba14-c0faec1b6794",
        "data": {
          "body": "Ffff",
          "date": "2022-11-07T09:10:13.573073"
        },
        "created_at": "2022-11-07T09:10:14.770503Z",
        "updated_at": "2022-11-07T09:10:14.770503Z"
      },
      "id": "fcecfa2a-5e7b-11ed-8080-80014ea006de",
      "object": {
        "id": "e147b773-d13d-4952-ba14-c0faec1b6794",
        "collection": "post",
        "foreign_id": "post:e147b773-d13d-4952-ba14-c0faec1b6794",
        "data": {
          "body": "Ffff",
          "date": "2022-11-07T09:10:13.573073"
        },
        "created_at": "2022-11-07T09:10:14.770503Z",
        "updated_at": "2022-11-07T09:10:14.770503Z"
      },
      "origin": null,
      "target": "",
      "time": "2022-11-07T09:10:13.573073",
      "user_id": "95db56be-2c60-46d6-8b4a-c2f71d7e666e",
      "verb": "post"
    }
  ],
  "next": "/api/v1.0/enrich/feed/PROFILE/95db56be-2c60-46d6-8b4a-c2f71d7e666e/?api_key=q8jk2pkj8fur&id_lt=fcecfa2a-5e7b-11ed-8080-80014ea006de&limit=5",
  "duration": "12.94ms"
}
@followthemoney1 followthemoney1 added the enhancement New feature or request label Nov 9, 2022
@lifeShadow59
Copy link

@followthemoney1 is there any update?

because I'm facing the same issue.

@followthemoney1
Copy link
Author

@lifeShadow59 as I provide, u can use the same fix by forking your library

@lifeShadow59
Copy link

Thanks @followthemoney1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants