DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/25/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
TITLE
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. See MPEP 606.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Instant Application: 19/190,554
U.S. Patent No. 12,316,973
1. A method for displaying large-format virtual reality content on a nonvirtual reality device comprising: identifying a location chosen by an operator comprising determining a point of interest at the location and picking a frame of action at the location to be provided in relation to the point of interest, the location is such that the point of interest and the frame of action are captured with a camera at the location, the point of interest to be accessible by a viewer in an interactive environment on a non-virtual reality device; recording a large-format content of the frame of action at the location; clarifying the large-format content into a finished large-format photo or video file, the clarifying comprising photo or video sorting wherein the large-format content is sorted to subsequently undergo stitching to derive a large-format or equirectangular photo or video, the large-format or equirectangular photo or video then undergoing adjusting which results in the finished large-format photo video file; and reframing the finished large-format photo or video file wherein the reframing comprises manipulating the finished large-format photo or video file into a standard aspect ratio photo or video for displaying in the interactive environment viewable on the non-virtual reality device such that a limited viewable area is exported as a standard aspect ratio configuration.
1. A method for displaying 360-degree virtual reality video on a non-[[ ]]virtual reality device comprising: identifying a location chosen by an operator[[-]] comprising determining a point of interest at the location and picking a frame of action at the location to be provided in relation to the point of interest, the location is such that the point of interest and the frame of action are captured with a camera at the location, the point of interest to be accessible by a viewer in an interactive environment on a non-virtual reality device; recording a 360-degree video of the frame of action at the location; and clarifying the 360-degree video into a finished 360-degree video file, the clarifying comprising video sorting wherein the 360-degree video is sorted to subsequently undergo video stitching to derive an equirectangular video, the equirectangular video then undergoing video adjusting which results in the finished 360-degree video file, andframing reframing the finished 360-degree video file wherein the framing reframing comprises manipulating the finished 360-degree video file into a standard aspect ratio video for displaying in the interactive environment viewable on the non-virtual reality device by manually moving a virtual camera via a video editing software to limit the viewable area to follow a chosen visual action such that the limited viewable area is exported as a standard aspect ratio video.
4. The method of claim 1, wherein a first camera is used for shooting both a large-format photos of the point of interest and used for recording the large-format video of the frame of action.
5. The method of claim 15, wherein a first camera is used for shooting both the 360-degree photos of the point of interest and used for recording the 360-degree video of the frame of action.
5. The method of claim 4, wherein the first camera simultaneously performs shooting and recording functions.
6. The method of claim 5, wherein the first camera simultaneously performs shooting and recording functions.
6. The method of claim 4, wherein a first camera is used for shooting the large-format photos of the point of interest at the location and a second camera is used for recording the large- format video of the frame of action at the location.
7. The method of claim 15, wherein a first camera is used for shooting the 360-degree photos of the point of interest at the location and a second camera is used for recording the 360-degree video of the frame of action at the location.
7. The method of claim 1, wherein the clarifying is further comprised of organizing multiple large-format or equirectangular photos or videos into a storyboard sequence of the frame of action at the location.
8. The method of claim 1, wherein the clarifying is further comprised of organizing multiple equirectangular videos into a storyboard sequence of the frame of action at the location.
8. The method of claim 1, further comprising shooting one or more large-format photos of the point of interest at the location.
14. The method of claim 1, further comprising shooting one or more 360-degree photos of the point of interest at the location.
9. The method of claim 8, wherein the large-format video of the frame of action provides content beyond the large-format photos of the point of interest.
15. The method of claim 14, wherein the 360-degree video of the frame of action provides content beyond the 360-degree photos of the point of interest.
10. The method of claim 9, further comprising refining the large-format photos into a finished photo file, the refining comprising photo sorting wherein the large-format photos are sorted to subsequently undergo photo stitching to derive a large-format or equirectangular photo, the large-format or equirectangular photo adjusted to result in the finished photo file.
16. The method of claim 14, further comprising refining the 360- degree photos into a finished photo file, the refining comprising photo sorting wherein the 360-degree photos are sorted to subsequently undergo photo stitching to derive an equirectangular photo, the equirectangular photo adjusted to result in the finished photo file.
11. The method of claim 10, further comprising: compiling the finished video file and the finished photo file, the finished video file and the finished photo file combined to be displayed in the interactive environment, the compiling comprising uploading, embedding, and publishing,wherein the uploading comprises uploading the finished video file and the finished photo file to a hosting server, the finished photo or video file displayed as a background to the interactive environment, wherein the embedding comprises the finished photo or video file being embedded to a particular spot on the background,wherein publishing comprises allowing the viewer access to the interactive environment.
17. The method of claim 16, further comprising compiling the finished video file and the finished photo file, the finished video file and the finished photo file combined to be displayed in the interactive environment, the compiling comprising uploading, embedding, and publishing,wherein the uploading comprises uploading the finished video file and the finished photo file to a hosting server, the finished photo file displayed as a background to the interactive environment,wherein the embedding comprises the finished video file being embedded to a particular spot on the background,wherein publishing comprises allowing the viewer access to the interactive environment.
12. The method of claim 8, wherein the shooting comprises extracting a singular still image frame from the large-format content which serves as a background where the reframed, standard aspect ratio photo or video is re-inserted.
18. The method of claim 14, wherein the shooting comprises extracting still frames a singular still image frame from the 360-degree video which serves as a background where the refrained, standard aspect ratio video is re-inserted.
13. The method of claim 12, further comprising: compiling the finished photo or video file, wherein compiling comprises embedding, overlaying,and publishing,wherein the embedding comprises the finished photo or video file being embedded to a particular spot on a background to the interactive environment, the spot being the location of the frame of action to provide more information about content communicated in the interactive environment,wherein the overlaying comprises overlaying additional information in the interactive environment, the point of interest identifiable by the viewer interacting with the interactive environment as a selectable space of content in the interactive environment,wherein the publishing comprises allowing the viewer access to the interactive environment on the non-virtual reality device such that the interactive environment is displayed as a large-format content upon which standard aspect ratio photo or video originating from the large-format content is displayed.
20. The method of claim 1, further comprising: compiling the finished video file, wherein compiling comprises embedding, overlaying, andpublishing,wherein the embedding comprises the finished video file being embedded to a particular spot on a background to the interactive environment, the spot being the location of the frame of action to provide more information about content communicated in the interactive environment,wherein the overlaying comprises overlaying additional information in the interactive environment, the point of interest identifiable by the viewer interacting with the interactive environment as a selectable space of content in the interactive environment,wherein the publishing comprises allowing the viewer access to the interactive environment on the non-virtual reality device such that the interactive environment is displayed as a 360-degree photos upon which standard aspect ratio videos originating from the 360-degree videos are displayed.
14. The method of claim 13, wherein a large-format file content is displayed as the background to the interactive environment, the finished photo or video file embedded to a particular spot on the background as the point of interest, the point of interest is where a viewer may interact to access the additional information.
21. The method of claim 20, wherein a finished photo file is displayedas the background to the interactive environment, the finished video file embedded to a particular spot on the background as the point of interest, the point of interest is where a viewer may interact to access the additional information.
15. The method of claim 14, wherein the large-format file content is displayed as the background to the interactive environment, the finished photo or video file embedded to a particular spot on the background as the point of interest, the point of interest is selectable by a user interacting with the interactive environment.
22. The method of claim 21, wherein the finished photo file is displayed as the background to the interactive environment, the finished video file embedded to a particular spot on the background as the point of interest, the point of interest is selectable by a user interacting with the interactive environment.
16. The method of claim 15, wherein the frame of action provides a pop-out window beyond the point of interest.
23. The method of claim 22, wherein the frame of action provides a pop-out window beyond the point of interest.
17. The method of claim 16, wherein the frame of action is an area on the large-format content.
24. The method of claim 23, wherein the frame of action is an area on the 360-degree photos.
18. The method of claim 16, wherein the frame of action provides text, images, or video.
25. The method of claim 24, wherein the frame of action provides text, images, or video.
19. The method of claim 16, wherein the frame of action is provided in relation to the point of interest.
27. The method of claim 24, wherein the frame of action is provided in relation to the point of interest.
20. The method of claim 19, wherein the point of interest is shot as a large-format photo and the frame of action is recorded as a large-format video.
28. The method of claim 24, wherein the point of interest is shot as a 360-degree photos and the frame of action is recorded as a 360-degree video.
21. The method of claim 20, wherein the large-format video of the frame of action provides content beyond the large-format photos of the point of interest.
29. The method of claim 28, wherein the 360-degree video of the frame of action provides content beyond the 360-degree photos of the point of interest.
22. The method of claim 20, further comprising extracting one or more large-format photos of the point of interest at the location from the large-format video of the frame of action.
30. The method of claim 24, further comprising extracting one or more 360-degree photos of the point of interest at the location from the 360-degree video of the frame of action.
23. The method of claim 13, wherein photo stitching comprises combining large-format photos into a large-format or equirectangular format.
31. The method of claim 13, wherein photo stitching comprises combining 360-degree photos into a 360-degree equirectangular format.
24. The method of claim 23, wherein the frame of action provides a pop-out window beyond the point of interest, the frame of action is an area on the large-format photos provided in relation to the point of interest.
32. The method of claim 31, wherein the frame of action provides a pop-out window beyond the point of interest, the frame of action is an area on the 360- degree photos provided in relation to the point of interest.
Claims 1 and 4-24 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 5-8, 14-18, 20-25 and 27-32 of U.S. Patent No. 12,316,973. Although the claims at issue are not identical, they are not patentably distinct from each other because they both provide large format virtual reality content on a non-virtual reality device through very similar claim language as seen above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8-10, 12-13 and 23-27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. (“Bendale”) (U.S. PG Publication No. 2018/0341811) in view of Stimm et al. (“Stimm”) (U.S. PG Publication No. 2021/0082085) and Shinohara et al. (“Shin”) (U.S. PG Publication No. 2022/0180491), in further view of Townsend et al. (“Town”) (U.S. Patent No. 11,158,344).
Regarding claim 1, Bendale teaches a method for displaying large-format virtual reality content on a nonvirtual reality device comprising:
identifying a location chosen by an operator (See ¶0071 wherein a location of capture of the 360-degree video may be identified, in this example a location may be a shopping mall; also see ¶0031 and 0056 wherein a location of the event may be annotated in the video, while a recognition engine may extract at least a location, orientation and pose of the camera as seen in ¶0062, wherein it is further understood by one of ordinary skill in the art that the location was chosen by the operator, especially since the system needs to be set up as it may comprise of variety of different components) comprising determining a point of interest at the location (See ¶0071 in view of 0031, 0056 and 0062 as described above) and picking a frame of action at the location to be provided in relation to the point of interest (Given the broadest reasonable interpretation consistent with applicant’s specification as seen in ¶0024 as originally filed, the frame of action may be taught as an object of interest, see ¶0040 and 0073 wherein objects or persons of interest may be selected based at least on a first or second user’s interaction [see ¶0074 for more information on the interactions of the user], or may itself be annotated by the system and may thus already have pre-generated metadata of the scenes or objects of interest as seen in ¶0071 and 0076), the location is such that the point of interest and the frame of action are captured with a camera at the location (See ¶0071), the point of interest to be accessible by a viewer in an interactive environment (See ¶0071-0073) on a non-virtual reality device;
recording a large-format content of the frame of action at the location (See ¶0071-0073 wherein the video may be [pre-]recorded or live);
clarifying the large-format content into a finished large-format photo or video file (See FIG. 2 in view of ¶0031, 0043 wherein the captured video from the camera system may be transmitted to an AR engine for processing the media content), the clarifying comprising photo or video sorting wherein the large-format content is sorted to subsequently undergo stitching to derive a large-format or equirectangular photo or video (See ¶0045 wherein the video may take the form of an equirectangular format, and when sent to a user device the video may undergo a destitching process as seen in ¶0046-0048 in order for the video to be viewed, thus inferring to the video undergoing a stitching process when previously converted into the equirectangular format).
Bendale, however, fails to teach the point of interest to be accessible by a viewer in an interactive environment on a non-virtual reality device; the large-format or equirectangular photo or video then undergoing adjusting which results in the finished large-format photo video file; and reframing the finished large-format photo or video file wherein the reframing comprises manipulating the finished large-format photo or video file into a standard aspect ratio photo or video for displaying in the interactive environment viewable on the non-virtual reality device such that a limited viewable area is exported as a standard aspect ratio configuration.
In a similar endeavor Stimm teaches the point of interest to be accessible by a viewer in an interactive environment on a non-virtual reality device (See ¶0107 wherein the content may be that of a 360 degree video [one that may even be converted via equirectangular projection], this is taken in view of ¶0126 wherein the computing device by which a viewer by view the displayed image data is seen as taught in ¶0126 which comprises various non-virtual reality devices).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Stimm into Bendale because it allows for viewing of 360 degree video content not only on a virtual-reality headset as described in ¶0107, but also on a screen on display device of almost any other kind as described in ¶0126, thus not limiting the scope of devices in which playback may occur.
In a similar endeavor Shin teaches the large-format or equirectangular photo or video then undergoing adjusting which results in the finished large-format photo video file (See ¶0033 in view of FIG. 4 wherein the video may undergo a correction processing which includes equirectangular processing, followed by an adjustment processing as part of the overall correction processing [which is part of the stitching process], this is taken in view of Bendale’s teachings; also see ¶0037-0038 for more information).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Shin into Bendale because it allows for the correction of image processing during 360-degree image data which may have distortions simply due to the wide angle imaging, such as those from fisheye lenses as described in ¶0002, and conversion of such image data onto an aspect ratio which is more suitable for viewing which may seem “flat” as seen in FIG. 4, thus improving visional comprehension for a user.
In a similar endeavor Town teaches reframing the finished large-format photo or video file wherein the reframing comprises manipulating the finished large-format photo or video file into a standard aspect ratio photo or video for displaying in the interactive environment viewable on the non-virtual reality device such that a limited viewable area is exported as a standard aspect ratio configuration (See col. 12, li. 8-31 and col. 14, li. 13-32, this is taken in view of Stimm).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Town into Bendale because it allows for an aspect ratio proportional to a viewing device as described in col. 12, li. 8-31, thus allowing for comfortable viewing to a viewer which may feel more natural in terms of displays.
Regarding claim 8, Bendale teaches the method of claim 1, further comprising shooting one or more large-format photos of the point of interest at the location (“In particular embodiments, the camera system 202 may capture one or more frames of a media content including, for example, images (including still images and/or video) and/or audio. Each frame of the media content may correspond to at least part of an image captured by one or more cameras,” - ¶0030, see ¶0030 and 0062).
Regarding claim 9, Bendale teaches the method of claim 8, wherein the large-format video of the frame of action provides content beyond the large-format photos of the point of interest (See for example ¶0031 wherein additional content may comprise metadata and annotations in various forms).
Regarding claim 10, Bendale fails to teach the method of claim 9, further comprising refining the large-format photos into a finished photo file, the refining comprising photo sorting wherein the large-format photos are sorted to subsequently undergo photo stitching to derive a large-format or equirectangular photo, the large-format or equirectangular photo adjusted to result in the finished photo file.
That is, while Bendale teaches the video being in an equirectangular format in ¶0045, wherein the system turns it into such formats through the AR engine as seen in FIG. 2 and part of such formats involving stitching through inference as described in ¶0046 wherein stitching was performed, such techniques are not taught by Bendale to be performed on the captured photos [still images].
In a similar endeavor Shin teaches further comprising refining the large-format photos into a finished photo file, the refining comprising photo sorting wherein the large-format photos are sorted to subsequently undergo photo stitching to derive a large-format or equirectangular photo, the large-format or equirectangular photo adjusted to result in the finished photo file (See ¶0033 in view of FIG. 4 wherein video may undergo a correction processing which includes equirectangular processing, followed by an adjustment processing as part of the overall correction processing [which is part of the stitching process], it is obvious to one of ordinary skill in the art that if this processing is done for video imaging data then such a system would be enabled to apply such techniques to still image data, especially since video image data is a combination of still image data; also see ¶0037-0038 for more information).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Shin into Bendale because it allows for the correction of image processing during 360-degree image data which may have distortions simply due to the wide angle imaging, such as those from fisheye lenses as described in ¶0002, and conversion of such image data onto an aspect ratio which is more suitable for viewing which may seem “flat” as seen in FIG. 4, thus improving visional comprehension for a user.
Regarding claim 12, Bendale teaches the method of claim 8, wherein the shooting comprises extracting a singular still image frame from the large-format content which serves as a background where the reframed, standard aspect ratio photo or video is re-inserted (See ¶0030-0031 and 0071-0074).
Regarding claim 13, Bendale teaches the method of claim 12, further comprising:
compiling the finished photo or video file, wherein compiling comprises embedding, overlaying, and publishing,
wherein the embedding comprises the finished photo or video file being embedded to a particular spot on a background to the interactive environment, the spot being the location of the frame of action to provide more information about content communicated in the interactive environment, wherein the overlaying comprises overlaying additional information in the interactive environment, the point of interest identifiable by the viewer interacting with the interactive environment as a selectable space of content in the interactive environment.
Bendale, however, fails to teach wherein the publishing comprises allowing the viewer access to the interactive environment on the non-virtual reality device such that the interactive environment is displayed as a large-format content upon which standard aspect ratio photo or video originating from the large-format content is displayed.
In a similar endeavor Town and Stimm together teach wherein the publishing comprises allowing the viewer access to the interactive environment on the non-virtual reality device such that the interactive environment is displayed as a large-format content upon which standard aspect ratio photo or video originating from the large-format content is displayed (See col. 12, li. 8-31 of Town in view of the teachings of Stimm as seen above).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Stimm and Town into Bendale because it allows for viewing of 360 degree video content not only on a virtual-reality headset as described in ¶0107 of Stimm, but also on a screen on display device of almost any other kind as described in ¶0126 of Stimm, thus not limiting the scope of devices in which playback may occur, additionally it allows for an aspect ratio proportional to a viewing device as described in col. 12, li. 8-31, thus allowing for comfortable viewing to a viewer which may feel more natural in terms of displays.
Regarding claim 23, Bendale teaches the method of claim 13, wherein photo stitching comprises combining large-format photos into a large-format or equirectangular format (See ¶0045-0046).
Regarding claim 24, Bendale teaches the method of claim 23, wherein the frame of action provides a pop-out window beyond the point of interest, the frame of action is an area on the large-format photos provided in relation to the point of interest (See ¶0031-0033).
Regarding claim 25, Bendale teaches the method of claim 1, further comprising embedding an interactive 3D model corresponding to an object visible from the original large-format content, wherein the 3D model aligns spatially with the original camera's perspective (See ¶0031 and 0071-0074).
Regarding claim 26, Bendale fails to teach the method of claim 1, wherein the large-format content is generated entirely or partially by artificial intelligence.
In a similar endeavor Shin teaches wherein the large-format content is generated entirely or partially by artificial intelligence (See ¶0074-0076 and 0094).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Shin into Bendale because it allows for the correction of image processing during 360-degree image data which may have distortions simply due to the wide angle imaging, such as those from fisheye lenses as described in ¶0002, and conversion of such image data onto an aspect ratio which is more suitable for viewing which may seem “flat” as seen in FIG. 4, thus improving visional comprehension for a user.
Regarding claim 27, Bendale teaches the method of claim 1, wherein one or more steps selected from sorting, stitching, adjusting, embedding, or compiling are performed automatically by artificial intelligence or software algorithms without manual intervention (See ¶0043).
Claim(s) 4, 5, 11 and 14-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. (“Bendale”) (U.S. PG Publication No. 2018/0341811) in view of Shinohara et al. (“Shin”) (U.S. PG Publication No. 2022/0180491) and Stimm et al. (“Stimm”) (U.S. PG Publication No. 2021/0082085), in further view of Townsend et al. (“Town”) (U.S. Patent No. 11,158,344) and Ohtsuka et al. (“Oh”) (U.S. PG Publication No. 2022/0100571).
Regarding claim 4, Bendale fails to teach the method of claim 1, wherein a first camera is used for shooting both a large-format photos of the point of interest and used for recording the large-format video of the frame of action.
In a similar endeavor Oh teaches wherein a first camera is used for shooting both a large-format photos of the point of interest and used for recording the large-format video of the frame of action (See ¶0059 wherein the imaging device 13 may be a wide-angle image capturing device capable of spherical [360 degree] panoramic imaging and may be used to obtain a still image, a video image or both of the video and still image data).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Oh into Bendale because it allows for image capturing of a wider range of types, i.e. not simply just the video data, but image stills as well as described in ¶0059, thus improving the overall aspect of the system.
Regarding claim 5, Bendale teaches the method of claim 4, wherein the first camera simultaneously performs shooting and recording functions.
That is, Bendale fails to specify that the captured still images and/or video as taught in ¶0030 are done simultaneously.
In a similar endeavor Oh teaches wherein the first camera simultaneously performs shooting and recording functions (See ¶0059 wherein the imaging device 13 may be a wide-angle image capturing device capable of spherical [360 degree] panoramic imaging and may be used to obtain a still image, a video image or both of the video and still image data).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Oh into Bendale because it allows for image capturing of a wider range of types, i.e. not simply just the video data, but image stills as well as described in ¶0059, thus improving the overall aspect of the system.
Regarding claim 11, Bendale teaches the method of claim 10, further comprising: compiling the finished video file and the finished photo file, the finished video file and the finished photo file combined to be displayed in the interactive environment, the compiling comprising uploading, embedding, and publishing, wherein the uploading comprises uploading the finished video file and the finished photo file to a hosting server, the finished photo or video file displayed as a background to the interactive environment, wherein the embedding comprises the finished photo or video file being embedded to a particular spot on the background, wherein publishing comprises allowing the viewer access to the interactive environment.
That is, Bendale does indeed teach that the produced image/video files may be displayed to the user in the interactive environment, for example as an annotation as seen in ¶0031, while uploading may be seen in FIG. 1 and 2 wherein it is sent to a network or cloud [hosting server], embedding is done in the form of annotations and metadata as described in at least ¶0031 and 0071 as users may also share, create and modify interactive content on the 360-degree video, as such publishing is also taught as seen in ¶0071-0074 wherein the system allows viewers access to the interactive environment through various interactions. Bendale simply fails to show doing this to both “finished video files” and for “finished photo files”.
However, as described above, Oh teaches in ¶0059 shooting both the 360-degree photos and recording of the video together, while Shin teaches producing a finished photo file as seen in ¶0033 in view of FIG. 4 and described above wherein the photo files may also undergo a finishing process.
Thus, in view of the teachings above by Oh and Shin and via incorporation, Bendale teaches further comprising compiling the finished video file and the finished photo file, the finished video file and the finished photo file combined to be displayed in the interactive environment, the compiling comprising uploading, embedding, and publishing, wherein the uploading comprises uploading the finished video file and the finished photo file to a hosting server, the finished photo or video file displayed as a background to the interactive environment, wherein the embedding comprises the finished photo or video file being embedded to a particular spot on the background, wherein publishing comprises allowing the viewer access to the interactive environment (See the teachings of Oh and Shin in view of ¶0031, 0071-0074 and FIG. 1-2 of Bendale as described above).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Oh and Shin into Bendale because it allows for image capturing of a wider range of types, i.e. not simply just the video data, but image stills as well as described in ¶0059 of Oh, thus improving the overall aspect of the system, also it allows for the correction of image processing during 360-degree image data which may have distortions simply due to the wide angle imaging, such as those from fisheye lenses as described in ¶0002 of Shin, and conversion of such image data onto an aspect ratio which is more suitable for viewing which may seem “flat” as seen in FIG. 4 of Shin, thus improving visional comprehension for a user.
Regarding claim 14, Bendale fails to teach the method of claim 13, wherein a large-format file content is displayed as the background to the interactive environment, the finished photo or video file embedded to aparticular spot on the background as the point of interest, the point of interest is where a viewer may interact to access the additional information.
That is, Bendale does indeed teach that the produced image/video files may be displayed to the user in the interactive environment, for example as an annotation as seen in ¶0031, while uploading may be seen in FIG. 1 and 2 wherein it is sent to a network or cloud [hosting server], embedding is done in the form of annotations and metadata as described in at least ¶0031 and 0071 as users may also share, create and modify interactive content on the 360-degree video, as such publishing is also taught as seen in ¶0071-0074 wherein the system allows viewers access to the interactive environment through various interactions. Bendale simply fails to show doing this to both “finished video files” and for “finished photo files”.
However, as described above, Oh teaches in ¶0059 shooting both the 360-degree photos and recording of the video together, while Shin teaches producing a finished photo file as seen in ¶0033 in view of FIG. 4 and described above wherein the photo files may also undergo a finishing process.
Thus, in view of the teachings above by Oh and Shin and via incorporation, Bendale teaches wherein a finished photo file is displayed as the background to the interactive environment (See the teachings of Oh and Shin in view of ¶0031, 0071-0074 and FIG. 1-2 of Bendale as described above), the finished video file embedded to a particular spot on the background as the point of interest (See ¶0031 and 00071-0074 and FIG. 1-2 of Bendale), the point of interest is where a viewer may interact to access the additional information (See ¶0031 and 00071-0074 and FIG. 1-2 of Bendale).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Oh and Shin into Bendale because it allows for image capturing of a wider range of types, i.e. not simply just the video data, but image stills as well as described in ¶0059 of Oh, thus improving the overall aspect of the system, also it allows for the correction of image processing during 360-degree image data which may have distortions simply due to the wide angle imaging, such as those from fisheye lenses as described in ¶0002 of Shin, and conversion of such image data onto an aspect ratio which is more suitable for viewing which may seem “flat” as seen in FIG. 4 of Shin, thus improving visional comprehension for a user.
Regarding claim 15, Bendale teaches the method of claim 14, wherein the large-format file content is displayed as the background to the interactive environment, the finished photo or video file embedded to a particular spot on the background as the point of interest, the point of interest is selectable by a user interacting with the interactive environment (See ¶0030-0033 wherein they may take the form of annotations or other metadata related items which may be interactable with users).
Regarding claim 16, Bendale teaches the method of claim 15, wherein the frame of action provides a pop-out window beyond the point of interest (See ¶0031-0033).
Regarding claim 17, Bendale teaches the method of claim 16, wherein the frame of action is an area on the large-format content (Given the broadest reasonable interpretation consistent with applicant’s specification as seen in ¶0024 as originally filed, the frame of action may be taught as an object of interest, see ¶0040 and 0073 wherein objects or persons of interest may be selected based at least on a first or second user’s interaction [see ¶0074 for more information on the interactions of the user], or may itself be annotated by the system and may thus already have pre-generated metadata of the scenes or objects of interest as seen in ¶0071 and 0076).
Regarding claim 18, Bendale teaches the method of claim 16, wherein the frame of action provides text, images, or video (See ¶0031-0033).
Regarding claim 19, Bendale teaches the method of claim 16, wherein the frame of action is provided in relation to the point of interest (See ¶0031-0033).
Claim(s) 6 and 20-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. (“Bendale”) (U.S. PG Publication No. 2018/0341811) in view of Shinohara et al. (“Shin”) (U.S. PG Publication No. 2022/0180491) and Stimm et al. (“Stimm”) (U.S. PG Publication No. 2021/0082085), in further view of Townsend et al. (“Town”) (U.S. Patent No. 11,158,344) and Neeter (U.S. PG Publication No. 2018/0061138).
Regarding claim 6, Bendale fails to teach the method of claim 4, wherein a first camera is used for shooting the large-format photos of the point of interest at the location and a second camera is used for recording the large- format video of the frame of action at the location.
In a similar endeavor Neeter teaches wherein a first camera is used for shooting the large-format photos of the point of interest at the location and a second camera is used for recording the large- format video of the frame of action at the location (See ¶0025 in view of FIG. 1 wherein the system may comprise a plurality of sensors, of which examples may include cameras for capture of still images [including in 360 degrees] and well as cameras for video capture [also including 360 degrees]).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Neeter into Bendale because it allows for the implementation of multiple types of sensors into a system as seen in FIG. 1 in view of ¶0025, each of which may obtain real world parameters such as still image data, video image data, audio data, osmic data, haptic data, etc., thus improving the capabilities of the overall system if more real world parameter data is needed.
Regarding claim 20, Bendale teaches the method of claim 19, wherein the point of interest is shot as a large-format photo and the frame of action is recorded as a large-format video.
In a similar endeavor Neeter teaches wherein the point of interest is shot as a large-format photo and the frame of action is recorded as a large-format video (See ¶0025 in view of FIG. 1 wherein the system may comprise a plurality of sensors, of which examples may include cameras for capture of still images [including in 360 degrees] and well as cameras for video capture [also including 360 degrees]).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Neeter into Bendale because it allows for the implementation of multiple types of sensors into a system as seen in FIG. 1 in view of ¶0025, each of which may obtain real world parameters such as still image data, video image data, audio data, osmic data, haptic data, etc., thus improving the capabilities of the overall system if more real world parameter data is needed.
Regarding claim 21, Bendale teaches the method of claim 20, wherein the large-format video of the frame of action provides content beyond the large-format photos of the point of interest (See ¶0031-0033).
Regarding claim 22, Bendale teaches the method of claim 20, further comprising extracting one or more large-format photos of the point of interest at the location from the large-format video of the frame of action (See ¶0031-0033).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. (“Bendale”) (U.S. PG Publication No. 2018/0341811) in view of Shinohara et al. (“Shin”) (U.S. PG Publication No. 2022/0180491) and Stimm et al. (“Stimm”) (U.S. PG Publication No. 2021/0082085), in further view of Townsend et al. (“Town”) (U.S. Patent No. 11,158,344) and Williams et al. (“Williams”) (U.S. PG Publication No. 2009/0007202).
Regarding claim 7, Bendale fails to teach the method of claim 1, wherein the clarifying is further comprised of organizing multiple large-format or equirectangular photos or videos into a storyboard sequence of the frame of action at the location.
In a similar endeavor Williams teaches wherein the clarifying is further comprised of organizing multiple large-format or equirectangular photos or videos into a storyboard sequence of the frame of action at the location (See ¶0098-0102 and 0004 in view of FIG. 7 wherein videos may be organized into vignettes and summary representations, wherein it is understood by one of ordinary skill in the art that this could comprise the frame of action from within the video data).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Williams into Bendale because it allows for organizing of video data into vignettes and summaries as seen in FIG. 7 which itself allows for more organization of video data, thus improving the overall system.
Claim(s) 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bendale et al. (“Bendale”) (U.S. PG Publication No. 2018/0341811) in view of Shinohara et al. (“Shin”) (U.S. PG Publication No. 2022/0180491) and Stimm et al. (“Stimm”) (U.S. PG Publication No. 2021/0082085), in further view of Townsend et al. (“Town”) (U.S. Patent No. 11,158,344) and Nanno et al. (“Nanno”) (U.S. PG Publication No. 2018/0007404).
Regarding claim 28, Bendale fails to teach the method of claim 1, wherein reframing comprises generating an alpha channel video configured to be transparently embedded onto a background image derived from the large-format content.
In a similar endeavor Nanno teaches wherein reframing comprises generating an alpha channel video configured to be transparently embedded onto a background image derived from the large-format content (See ¶0066 in view of FIG. 9).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Nanno into Bendale because it allows for the insertion of an image to be embeddd through an alpha channel as described in ¶0066, which allows it to be partially transparent that allows for a variety of options as described in at least ¶0038.
Allowable Subject Matter
Claims 2 and 3 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
That is, prior art was found in the form of Cregan et al. (U.S. Patent No. 10,489,965) which discloses a wayfinding system that allows users to manually manipulate the view they are presented through a virtual camera position and orientation as described in col. 17, li. 39-53 and col. 18, li. 42-53, but fails to explicitly cite that such manual manipulation is additionally a limited viewable area being exported as a standard aspect ratio configuration from the overall large-format view photo or video. Similarly, an AI system was not found to do this in view of all of the limitations of the independent claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDEMIO NAVAS JR whose telephone number is (571)270-1067. The examiner can normally be reached M-F, ~ 9 AM -6 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at 5712727383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EDEMIO NAVAS JR
Primary Examiner
Art Unit 2483
/EDEMIO NAVAS JR/Primary Examiner, Art Unit 2426