DETAILED ACTION
This Office action is in response to the communication filed on November 13, 2025. Claims 1-22 remain pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to amended claims 1, 10, and 18 in the Remarks section (pages 6-9, dated 11/13/2025) have been fully considered but are moot because the arguments do not apply to the current combination of references being used in the current rejection.
Applicant argues Dvir does not teach EFOV that comprise a height map indicating a distance of each pixel of an image from a camera view, and therefore also does not teach a parallax pixel map comprising of a height map with dimensions encoded.
However, Dvir teaches creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. When this occurred, the height interpreted as the XYZ differential was changed by a particular distance in each of the X-, Y-, and Z-axes, the resulting rectangular was a 3-dimensional height map of each pixel following the rotation. An encoder could also present EFOV frame status information that included the dimensions, for example, the width, the height and/or the like of EFOV frame in the selected projection format, for example, the ERP format, the rectilinear format and/or the like. The resulting encoded rectangular after transformation would visually show the pixels at the distances that were annotated in axes of weight and height. If each individual pixel was labelled in the height map with its height, that limitation should be claimed with written subject matter support. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant argues Dvir in view of Vaittinen does not teach wherein the first current camara includes pixels that were obstructed from the perspective the first camera view by one or more pixels because Vaittinen does not teach a parallax pixel map comprising of a first scene and a height map and updating the parallax pixel map according to camera view including obstructed pixels. However, Vaittinen was not relied on to teach the parallax pixel map, and the combination of Dvir and Vaittinen is relied upon to teach the claim limitations. In page 45, Dvir teaches since the orientation of the client device changes, the current orientation data may not be aligned with updated current orientation data. One EFOV frame was adjusted such as cropped, shifted, zoomed in and zoomed out, in order to adapt to the updated current orientation data, Dvir did not specifically teach the consequence of the adjustment such as a shift of the field of view such as between a zoomed in and zoomed out EFOV frame. Vaittenen teaches particularly a mapping and user interface application 107 processes and/or facilitates a processing of location information associated with the one or more items to determine occlusion information with respect to one or more other objects of the user interface, the location-based service such as whether the item (e.g., a contact location or POI) is located so that the item would not be visible from the current perspective of the user interface (i.e., until the perspective was changed). Based, at least in part, on the occlusion information, the application 107 can apply different rendering techniques for one or more preview user interface objects used to represent the items in the rendered user interface. The pixels occluded from view were obstructed from view and the EPOV processing can process occlusion information as taught by Vaittinen. Therefore, meeting claim limitations. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant's arguments have been fully considered with respect to 2-9, 11-17, and 19-22 in the Remarks section (pages 8-9) but they are not persuasive as the claims depend upon the features recited in the amended independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8-13, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Foreign Patent Publication WO 2019120638 A1 by Dvir, for which the machine language translation is being used for the citations below, in view of U.S. Patent Publication 2012/0240077 A1 by Vaittinen.
Regarding claim 1, Dvir teaches a method comprising:
receiving, at a client device (Page 1, fifth paragraph. The present invention, in some embodiments thereof, relates to streaming Virtual Reality (VR) 360 video content to client devices), a parallax pixel map (Page 2, paragraphs 3-5. Segmenting a sphere defined in a VR video file projected in an equirectangular projection (ERP) format into a plurality of overlapping segments according to the EFOV width and the EFOV height, Page 51, second to fourth paragraphs. Similarly, the Applicant defines the parallax pixel map as the image and the height map in specification as filed page 2, lines 29-30), comprising
a first image of a scene and a height map wherein the first image comprises a plurality of pixels from a perspective of a first camera view (Page 51, second to fourth paragraphs. The EFOV width and the EFOV height may be defined according to values of the FOV frame, for example, FOV angles, i.e. a horizontal FOV angle value and a vertical FOV angle value), and wherein the height map indicates a distance of each pixel of the first image from the first camera view (Page 3, first and second paragraph and Page 55, first to third paragraphs. Creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. Also, see interpolation in page 56, last paragraph.);
updating the parallax pixel map based on a first current camera view of the scene from a current perspective of the client device to generate an updated parallax pixel map comprising at least one of an updated first image and an updated height map (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame of a group of pictures (GOP) of the VR video file.)
rendering a current display frame (150) based on the updated parallax pixel map (142);and providing the current display frame for display (Page 9, first paragraph. Therefore in case of switching the client device needs to use the IDR frames of the newly delivered EVP stream. By providing a plurality of IDR shifted EVP streams for each viewport (overlapping segment) the IDR shifted EVP stream having an IDR frame located within the GOP at the exact point in time of the switch may be provided to the client device.)
While in page 45, Dvir teaches since the orientation of the client device changes, the current orientation data may not be aligned with updated current orientation data. One EFOV frame was adjusted such as cropped, shifted, zoomed in and zoomed out, in order to adapt to the updated current orientation data, Dvir did not specifically teach the consequence of the adjustment such as a shift of the field of view such as between a zoomed in and zoomed out EFOV frame. Therefore, Dvir does not teach wherein the first current camera view includes pixels that were obstructed from the perspective of the first camera view by one or more pixels.
However in the analogous art of rendering field of view in augmented or virtual reality, Vaittinen teaches determining to render, at a device, a user interface for a location-based service based, at least in part, on a field of view. The method also comprises determining one or more items outside of the field of view. The method further comprises determining to render one or more user interface objects in the user interface. The one or more user interface objects present, at least in part, preview information associated with the one or more items (Vaittinen [0003] and [0026]).
When using a location-based user interface (e.g., a mixed reality view, an augmented reality view, a virtual reality view, and the like), users often desire to quickly jump between locations, items of interests (e.g., points of interests, locations of contacts, event locations, etc.), and the like. However, it can be a problem that switching between locations, items, etc. in a location-based user interface can cause confusion in the understanding of the space and location, especially when the user is not very familiar with the locations depicted in the user interface. The mapping and user interface application 107 processes and/or facilitates a processing of location information associated with the one or more items to determine occlusion information with respect to one or more other objects of the user interface, the location-based service, or a combination thereof (step 307). For example, the application 107 can determine whether the item (e.g., a contact location or POI) is located within or behind one or more buildings so that the item would not be visible from the current perspective of the user interface (i.e., until the perspective was changed). Based, at least in part, on the occlusion information, the application 107 can apply different rendering techniques for one or more preview user interface objects used to represent the items in the rendered user interface (Vaittinen [0003] and [0065]). It would have been obvious to have used a field of view that accounted for occlusion information as taught by Vaittinen in the field of view processing of Dvir. One having ordinary skill in the art would have been motivated to have occlusion information calculated for location-based user interfaces that were complex, to address technical challenges to presenting additional information (e.g., preview information about destinations, locations, or other items not within the current field of view) in location-based user interfaces for efficient access by users (Vaittinen [0001] and [0064]).
Regarding claim 2, Dvir of the combination of references further teaches the method of claim 1, wherein generating the current display frame comprises projecting the plurality of pixels of the updated parallax pixel map onto corresponding pixels of the first image at a frame buffer (Page 32, first paragraph and Page 31, last paragraph. The additional presentation data may serve as a buffer which may be used by the client device(s) 106 to locally generate updated FOV frames according to updated orientation data of the client device. Naturally, one or more of the users 110 may change the selected FOV. The cropped segment of the VR 360 video may therefore need to be updated accordingly, i.e. the encoding apparatus 102 may need to generate new VR 360 video segments for the client device(s) 106 reflecting the new FOV(s).)
Regarding claim 3, Dvir of the combination of references further teaches the method of claim 2, further comprising: identifying a portion of the parallax pixel map based on a portion of the scene that is visible from the first current camera view at the client device; and wherein projecting the parallax pixel map comprises projecting the portion of the parallax pixel map onto the first image to generate the current display frame (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame portion of a group of pictures (GOP) of the VR video file.)
Regarding claim 4, Dvir of the combination of references further teaches the method of claim 1,wherein updating the parallax pixel map comprises updating the height map based on a distance of each pixel of the first image from the first current camera view at the client device (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame of a group of pictures (GOP) of the VR video file from a previous viewport to a next viewport. Page 62, last paragraph and Page 63, first paragraph. The number of the IDR shifted EVP streams created for a respective EVP stream may therefore equal the number of frames in the GOP. As such the first IDR shifted EVP stream (1) has the IDR frame at frame position 1).
Regarding claim 5, Dvir of the combination of references further teaches the method claim 1,wherein updating the parallax pixel map is based on a change in rotation or position from the first camera view to the first current camera view (Page 31, last paragraph. 106. Naturally, one or more of the users 110 may change the selected FOV, for example, change a location of the center of FOV, increase the FOV (zoom-out), decrease the FOV (zoom-in) and/or the like.)
Regarding claim 8, Dvir of the combination of references further teaches the method of any preceding claim claim 1,wherein the further comprising information for rendering pixels that were not visible from the perspective of the first camera view (Page 32, first paragraph. To support this, the EVP streams generated by the encoding apparatus 102 for the cropped segment(s) of the VR 360 video include Extended FOV (EFOV, also referred to as FOV+) frames which encompass a larger FOV area than the FOV area presented to the user(s) 110 by the client device(s) 106).
Further in the analogous art of rendering field of view in augmented or virtual reality, Vaittinen teaches determining to render, at a device, a user interface for a location-based service based, at least in part, on a field of view. The method also comprises determining one or more items outside of the field of view. The method further comprises determining to render one or more user interface objects in the user interface. The one or more user interface objects present, at least in part, preview information associated with the one or more items (Vaittinen [0003] and [0026]).
When using a location-based user interface (e.g., a mixed reality view, an augmented reality view, a virtual reality view, and the like), users often desire to quickly jump between locations, items of interests (e.g., points of interests, locations of contacts, event locations, etc.), and the like. However, it can be a problem that switching between locations, items, etc. in a location-based user interface can cause confusion in the understanding of the space and location, especially when the user is not very familiar with the locations depicted in the user interface. The mapping and user interface application 107 processes and/or facilitates a processing of location information associated with the one or more items to determine occlusion information with respect to one or more other objects of the user interface, the location-based service, or a combination thereof (step 307). For example, the application 107 can determine whether the item (e.g., a contact location or POI) is located within or behind one or more buildings so that the item would not be visible from the current perspective of the user interface (i.e., until the perspective was changed). Based, at least in part, on the occlusion information, the application 107 can apply different rendering techniques for one or more preview user interface objects used to represent the items in the rendered user interface (Vaittinen [0003] and [0065]). It would have been obvious to have used a field of view that accounted for occlusion information as taught by Vaittinen in the field of view processing of Dvir. One having ordinary skill in the art would have been motivated to have occlusion information calculated for location-based user interfaces that were complex, to address technical challenges to presenting additional information (e.g., preview information about destinations, locations, or other items not within the current field of view) in location-based user interfaces for efficient access by users (Vaittinen [0001] and [0064]).
Regarding claim 9, Dvir of the combination of references further teaches the method of claim 1,further comprising: updating the parallax pixel map based on a second current camera view at the client device; generating a second current display frame based on the updated parallax pixel map; and providing the second current display frame for display at the client device (Page 62, last paragraph and Page 63, first paragraph. The number of the IDR shifted EVP streams created for a respective EVP stream may therefore equal the number of frames in the GOP. As such the first IDR shifted EVP stream (1) has the IDR frame at frame position 1, the second IDR shifted EVP stream (2) has the IDR frame at frame position 2.)
Regarding claim 10, Dvir teaches a method comprising:
receiving, at a client device (Page 1, fifth paragraph. The present invention, in some embodiments thereof, relates to streaming Virtual Reality (VR) 360 video content to client devices), a parallax pixel map (Page 2, paragraphs 3-5. Segmenting a sphere defined in a VR video file projected in an equirectangular projection (ERP) format into a plurality of overlapping segments according to the EFOV width and the EFOV height, Page 51, second to fourth paragraphs. Similarly, the Applicant defines the parallax pixel map as the image and the height map in specification as filed page 2, lines 29-30), comprising
an image of a scene from a first camera view comprising a plurality of pixels and a height map (Page 51, second to fourth paragraphs. The EFOV width and the EFOV height may be defined according to values of the FOV frame, for example, FOV angles, i.e. a horizontal FOV angle value and a vertical FOV angle value), and a height map indicating a distance of each pixel of the first image from the first camera view (Page 3, first and second paragraph and Page 55, first to third paragraphs. Creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. Also, see interpolation in page 56, last paragraph.); identifying a portion of the image of the parallax pixel map that is visible from a current camera view; projecting the portion of the parallax pixel map onto corresponding pixels of the image at a frame buffer to render a current display frame from the current camera (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted iew EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame portion of a group of pictures (GOP) of the VR video file. Page 32, first paragraph and Page 31, last paragraph. The additional presentation data may serve as a buffer which may be used by the client device(s) 106 to locally generate updated FOV frames according to updated orientation data of the client device).
While in page 45, Dvir teaches since the orientation of the client device changes, the current orientation data may not be aligned with updated current orientation data. One EFOV frame was adjusted such as cropped, shifted, zoomed in and zoomed out, in order to adapt to the updated current orientation data, Dvir did not specifically teach the consequence of the adjustment such as a shift of the field of view such as between a zoomed in and zoomed out EFOV frame. Therefore, Dvir does not teach wherein the first current camera view includes pixels that were obstructed from the perspective of the first camera view by one or more pixels.
However in the analogous art of rendering field of view in augmented or virtual reality, Vaittinen teaches determining to render, at a device, a user interface for a location-based service based, at least in part, on a field of view. The method also comprises determining one or more items outside of the field of view. The method further comprises determining to render one or more user interface objects in the user interface. The one or more user interface objects present, at least in part, preview information associated with the one or more items (Vaittinen [0003] and [0026]).
When using a location-based user interface (e.g., a mixed reality view, an augmented reality view, a virtual reality view, and the like), users often desire to quickly jump between locations, items of interests (e.g., points of interests, locations of contacts, event locations, etc.), and the like. However, it can be a problem that switching between locations, items, etc. in a location-based user interface can cause confusion in the understanding of the space and location, especially when the user is not very familiar with the locations depicted in the user interface. The mapping and user interface application 107 processes and/or facilitates a processing of location information associated with the one or more items to determine occlusion information with respect to one or more other objects of the user interface, the location-based service, or a combination thereof (step 307). For example, the application 107 can determine whether the item (e.g., a contact location or POI) is located within or behind one or more buildings so that the item would not be visible from the current perspective of the user interface (i.e., until the perspective was changed). Based, at least in part, on the occlusion information, the application 107 can apply different rendering techniques for one or more preview user interface objects used to represent the items in the rendered user interface (Vaittinen [0003] and [0065]). It would have been obvious to have used a field of view that accounted for occlusion information as taught by Vaittinen in the field of view processing of Dvir. One having ordinary skill in the art would have been motivated to have occlusion information calculated for location-based user interfaces that were complex, to address technical challenges to presenting additional information (e.g., preview information about destinations, locations, or other items not within the current field of view) in location-based user interfaces for efficient access by users (Vaittinen [0001] and [0064]).
Regarding claim 11, Dvir of the combination of references further teaches the method of claim 10, further comprising updating the parallax pixel map based on the current camera view at the client device to generate an updated parallax pixel map (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame of a group of pictures (GOP) of the VR video file.)
Regarding claim 12, Dvir of the combination of references further teaches the method of claim 10, wherein updating the parallax pixel map comprises updating the height map based on a distance of each pixel of the image of the scene from the current camera view (Page 3, first and second paragraph and Page 55, first to third paragraphs. Creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. Also, see interpolation in page 56, last paragraph. Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s).)
Regarding claim 13, Dvir of the combination of references further teaches the method of claim 11, wherein updating the parallax pixel map is based on a change in rotation or position from the first camera view to the current camera view (Page 3, first and second paragraph and Page 55, first to third paragraphs. Creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. Also, see interpolation in page 56, last paragraph.)
Regarding claim 16, Dvir of the combination of references further teaches the method of claim 1,wherein the parallax pixel map (120) is received from a server (102) having rendered the first image (122) based on the first camera view (Page 16, second paragraph. Delivering (streaming) the VR 360 video content, for example, a VR 360 video file and/or the like from an encoding apparatus, for example, a server, a computing node, a cluster of computing nodes, a cloud service and/or the like to a client device).
Regarding claim 17, Dvir of the combination of references further teaches the method of claim 1,wherein the parallax pixel map (120) is part of rendered game content streamed to the client device (Page 2, second paragraph. On-line streaming of such VR 360 video content is therefore highly desired as the market potential for such streaming is practically endless for a plurality of applications, ranging from gaming applications, through training and simulation applications to life saving medical applications and/or defense applications).
Regarding claim 18, Dvir teaches at a client device (Page 1, fifth paragraph. The present invention, in some embodiments thereof, relates to streaming Virtual Reality (VR) 360 video content to client devices), comprising: a central processing unit (CPU) (Page 47, The client device includes a processor 424) to receive a parallax pixel map (Page 2, paragraphs 3-5. Segmenting a sphere defined in a VR video file projected in an equirectangular projection (ERP) format into a plurality of overlapping segments according to the EFOV width and the EFOV height, Page 51, second to fourth paragraphs. Similarly, the Applicant defines the parallax pixel map as the image and the height map in specification as filed page 2, lines 29-30), comprising
an image of a scene from a first camera view comprising a plurality of pixels and a height map (Page 51, second to fourth paragraphs. The EFOV width and the EFOV height may be defined according to values of the FOV frame, for example, FOV angles, i.e. a horizontal FOV angle value and a vertical FOV angle value), and a height map indicating a distance of each pixel of the first image from the first camera view (Page 3, first and second paragraph and Page 55, first to third paragraphs. Creating a plurality of EVP streams for the VR video file by performing the following in each one of a plurality of iterations applying a rotation to each of the plurality of overlapping segments of the sphere. Following the rotation, the pixels of the EFOV frame sphere may be efficiently enclosed by a rectangular as known in the art. The pixels were rotated by a particular XYZ distance. Also, see interpolation in page 56, last paragraph.); and a graphics processing unit (GPU) (Page, 42, last paragraph) The processor(s) 404, homogenous or heterogeneous, may include one or more processors arranged for parallel processing, as clusters (graphic) and/or as one or more multi core processor(s)) to: update the parallax pixel map based on a current camera view of the scene from a current perspective of the client device to generate an updated parallax pixel map (Page 8, last paragraph and Page 9, first paragraph. The Instantaneous decoding refresh IDR shifted EVP streams may allow a smooth transition between EVP streams corresponding to adjacent viewports (overlapping segments) when switching between EVP streams for client device(s) in response to FOV change of the client device(s). Instantaneous decoding refresh (IDR) shifted viewport streams for each of the plurality of EVP streams by repeating each of the plurality of iterations starting in a consecutive frame of a group of pictures (GOP) of the VR video file);
project the updated parallax pixel map onto corresponding pixels of the image to generate a current display frame; and provide the current display frame for display (Page 9, first paragraph. Therefore in case of switching the client device needs to use the IDR frames of the newly delivered EVP stream. By providing a plurality of IDR shifted EVP streams for each viewport (overlapping segment) the IDR shifted EVP stream having an IDR frame located within the GOP at the exact point in time of the switch may be provided to the client device.)
While in page 45, Dvir teaches since the orientation of the client device changes, the current orientation data may not be aligned with updated current orientation data. One EFOV frame was adjusted such as cropped, shifted, zoomed in and zoomed out, in order to adapt to the updated current orientation data, Dvir did not specifically teach the consequence of the adjustment such as a shift of the field of view such as between a zoomed in and zoomed out EFOV frame. Therefore, Dvir does not teach wherein the first current camera view includes pixels that were obstructed from the perspective of the first camera view by one or more pixels.
However in the analogous art of rendering field of view in augmented or virtual reality, Vaittinen teaches determining to render, at a device, a user interface for a location-based service based, at least in part, on a field of view. The method also comprises determining one or more items outside of the field of view. The method further comprises determining to render one or more user interface objects in the user interface. The one or more user interface objects present, at least in part, preview information associated with the one or more items (Vaittinen [0003] and [0026]).
When using a location-based user interface (e.g., a mixed reality view, an augmented reality view, a virtual reality view, and the like), users often desire to quickly jump between locations, items of interests (e.g., points of interests, locations of contacts, event locations, etc.), and the like. However, it can be a problem that switching between locations, items, etc. in a location-based user interface can cause confusion in the understanding of the space and location, especially when the user is not very familiar with the locations depicted in the user interface. The mapping and user interface application 107 processes and/or facilitates a processing of location information associated with the one or more items to determine occlusion information with respect to one or more other objects of the user interface, the location-based service, or a combination thereof (step 307). For example, the application 107 can determine whether the item (e.g., a contact location or POI) is located within or behind one or more buildings so that the item would not be visible from the current perspective of the user interface (i.e., until the perspective was changed). Based, at least in part, on the occlusion information, the application 107 can apply different rendering techniques for one or more preview user interface objects used to represent the items in the rendered user interface (Vaittinen [0003] and [0065]). It would have been obvious to have used a field of view that accounted for occlusion information as taught by Vaittinen in the field of view processing of Dvir. One having ordinary skill in the art would have been motivated to have occlusion information calculated for location-based user interfaces that were complex, to address technical challenges to presenting additional information (e.g., preview information about destinations, locations, or other items not within the current field of view) in location-based user interfaces for efficient access by users (Vaittinen [0001] and [0064]).
Regarding claim 19, Dvir in view of Vaittinen renders obvious the claim limitations in consideration of the grounds of rejection of claim 4 above.
Regarding claim 20, Dvir in view of Vaittinen renders obvious the claim limitations in consideration of the grounds of rejection of claim 5 above.
Claims 6-7, 14-15, and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over U Foreign Patent Publication WO 2019120638 A1 by Dvir in view U.S. Patent Publication 2012/0240077 A1 by Vaittinen, and further in view of Foreign Patent Publication CN 109640066 B by Song et al. (“Song,”) for which the machine language translation will be used in the citations below.
Regarding claim 6, Dvir in view of Vaittinen does not teach the method of claim 1,further comprising: identifying blank pixels (815) of the first image that have no corresponding pixels of the updated parallax pixel map; and assigning values (820) to the blank pixels.
However, in the analogous art of generating depth images and computer vision, Song teaches virtual reality depth images with left and right camera stereoscopic shooting or field of view required depth images of relatively high precision (Song Page 2, paragraphs 3-4). In one embodiment, after the corresponding matching pixel point and the pixel point to be matched in the first image according to the to-be-matched pixel points in the first image and in the second image, determining a first depth image, the method further comprises: detecting whether there is a blank area in the first depth image, wherein the blank area to comprise a plurality of data value is 0 of the area of the pixel point;
determining that the first deep lower, obtaining blank area exists in the image data value of the pixel point in the first depth image non-blank area and the blank area;
data value of the pixel point according to the first depth image in the non-blank area and the blank area, the modified data value of the pixel point of the blank area (Song Page 27, paragraphs 2-7).
It would have been obvious before the effective filing date of the invention to have similarly assigned data values to blank areas based on the deeper, lower pixels with pixel values. One having ordinary skill in the art would have been motivated to have further claims a high-precision dense deep image, without blank areas, and prevent more depth images that are sparse, the resolution is not high, and that have relatively poor accuracy (Song Page 27, paragraphs 6-7 and Page 3, paragraphs 1-3).
Regarding claim 7, Dvir in view of Vaittinen does not teach the method of claim 6, wherein the values assigned to the blank pixels are based on one of: values of pixels adjacent to the blank pixels; or voxels based on an angle between the first camera view and the first current camera view.
ated parallax pixel map; and assigning values (820) to the blank pixels.
However, in the analogous art of generating depth images and computer vision, Song teaches virtual reality depth images with left and right camera stereoscopic shooting or field of view required depth images of relatively high precision (Song Page 2, paragraphs 3-4). In one embodiment, after the corresponding matching pixel point and the pixel point to be matched in the first image according to the to-be-matched pixel points in the first image and in the second image, determining a first depth image, the method further comprises: detecting whether there is a blank area in the first depth image, wherein the blank area to comprise a plurality of data value is 0 of the area of the pixel point;
determining that the first deep lower, obtaining blank area exists in the image data value of the pixel point in the first depth image non-blank area and the blank area;
data value of the pixel point according to the first depth image in the non-blank area and the blank area, the modified data value of the pixel point of the blank area (Song Page 27, paragraphs 2-7).
It would have been obvious before the effective filing date of the invention to have similarly assigned data values to blank areas based on the deeper, lower pixels with pixel values. One having ordinary skill in the art would have been motivated to have further claims a high-precision dense deep image, without blank areas, and prevent more depth images that are sparse, the resolution is not high, and that have relatively poor accuracy (Song Page 27, paragraphs 6-7 and Page 3, paragraphs 1-3).
Regarding claim 14, Dvir in view of Vaittinen and Song renders obvious the claim limitations in consideration of the grounds of rejection of claim 6 above.
Regarding claim 15, Dvir in view of Vaittinen and Song renders obvious the claim limitations in consideration of the grounds of rejection of claim 7 above.
Regarding claim 21, Dvir in view of Vaittinen and Song renders obvious the claim limitations in consideration of the grounds of rejection of claim 6 above.
Regarding claim 22, Dvir in view of Vaittinen and Song renders obvious the claim limitations in consideration of the grounds of rejection of claim 7 above.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHEEN I JAVED whose telephone number is (571)272-0825. The examiner can normally be reached on Mon-Fri 9:00 am-5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AMR AWAD can be reached on 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHEEN I JAVED/Examiner, Art Unit 2621
/AMR A AWAD/Supervisory Patent Examiner, Art Unit 2621
Prosecution Insights