DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 7/14/2026 has been entered.
Response to Amendment
Claims 1-19 are pending. Claims 1, 18, and 19 are amended.
Response to Arguments
Applicant's arguments filed 7/14/2026 have been fully considered but they are not persuasive. The newly amended portion of the claims 1, 18, and 19 are found in Curry reference, specifically in figs. 25-27. Broadly speaking, decision boxes, 2504, 2508, 2512, 2516, 2520, 2524, 2528, 2532, 2534, 2540…etc. in fig. 25 and corresponding analogues in figs. 26-27, can reasonably function as trigger. In response to the trigger shown in diamond shaped decision boxes of figs. 5-27, virtual image generation and/or 3D model of a captured subject is created in addition to outputting the virtual image, 3D model or combination thereof, which are understood implemented in other rectangular boxes shown in figs. 25-27.
For further details see the rejection below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, and 9-19 is/are rejected under 35 U.S.C. 102(a)(1) and/or 102(a)(2) as being anticipated by Curry (US 20100026809 A1).
Regarding claim 1, Curry discloses an information processing device (fig. 8, ¶0006, 30a-d) comprising: circuitry ( Image processing techniques such as triangulation can be used to help detect the location of different objects such as the ball in a game of play, ¶0055) configured to perform processing of outputting, in response to a trigger (decision boxes, 2504, 2508, 2512, 2516, 2520, 2524, 2528, 2532, 2534, 2540…etc. in fig. 25 and corresponding analogues in figs. 26-27 can reasonably function as trigger), at least one of
a virtual image generated based on of estimation information regarding a subject, the estimation information being generated based on at least one of a captured image or sensor information, or a three-dimensional model of the subject in a case where a live-action image captured by an imaging device is output (The camera recorded the position of the actor's movements via the motion capture dots. Then the 3D CG character was fitted to positions recorded by the motion capture dots in the scene later. In a similar way then one embodiment (viewing mode) uses the position sensors to capture movements of the players. Then in software 3D CG avatars are fitted to the skeletal stick figures created by the multiple sensors. One method that can additionally be used to create the 3D CG character avatars is triangulation 3D laser scanning. This allows the system to get more precise 3D models of the players. The players motion points (skeletal and muscles) are still difficult to model completely inside software, ¶0063. Also see ¶0048, ¶0083, ¶0124-0133.
FIG. 20B displays a sample end user Menu interface 2088 that the viewer can use to select the different types of viewing options. A first option is Traditional Viewing Option 2090, a second option is Manual Viewing Option 2092, a third option is Player Manual Viewing Option 2094, and a fourth option is Mixed Viewing Option 2096, ¶0195.
The Manual Viewing Option 2092 makes use of the first and second methods used to implement the seventh step 1218 of FIG. 12. In this viewing option, 3D environments of the game are constructed using the 3D model of the event created in software, and the player CG avatars created using the player position information. The first method then uses the 3D event in software for broadcasting the event. The second method combines live video signal imagery with the 3D game occurring from inside software. This is accomplished as described before using two different techniques, ¶0211.
The Player Manual Viewing Option 2094 described here is for the sport of American football. Likewise, the same methodology can be applied to other sports such as soccer, and basketball. This allows the end user to transition through each of the POV shots of each player in the game one by one. Once the end user selects option 2094, they will next be referred to the end user game viewing options screen 2100 shown in FIG. 21, ¶0215),
wherein the outputting is switched between modes of outputting the virtual image, outputting the three-dimensional model, or outputting both the virtual image and the three-dimensional model in response to the trigger (Options 1-3 in fig. 20b could reasonably be understood as meeting limitation of outputting of a virtual image and/or a 3D model of a subject. E.g., option 2092 (or option 4) makes use of the first and second methods used to implement the seventh step 1218 of FIG. 12. In this viewing option, 3D environments of the game are constructed using the 3D model of the event created in software, and the player CG avatars created using the player position information. Option 2094 (or option 3) on the other hand allows the end user to transition through each of the POV shots of each player in the game one by one. Once the end user selects option 2094, they will next be referred to the end user game viewing options screen 2100 shown in FIG. 21. Therefore, option 2094 could reasonably meet the limitation regarding outputting of “virtual image generated based on estimation information regarding a subject”, while option 2092 can reasonably be understood as meeting the limitation of “a three-dimensional model of the subject”. Also, as described before, mixed viewing option 2096 can reasonably meet the limitation of “outputting both the virtual image and the three-dimensional model in response to the trigger.”
See fig 20b, ¶0195, ¶211-0218.
Also, according to the definition of trigger being decision boxes, 2504, 2508, 2512, 2516, 2520, 2524, 2528, 2532, 2534, 2540…etc. in fig. 25 and corresponding analogues in figs. 26-27, virtual image generation and/or 3D model of a captured subject is created in addition to outputting the virtual image, 3D model or combination thereof, as recited in the claim, implemented in other rectangular boxes shown in figs. 25-27), and
wherein the trigger includes at least one of a number of captured images of the subject, a posture of the subject, or a movement state of the subject (decision boxes, 2504, 2508, 2512, 2516, 2520, 2524, 2528, 2532, 2534, 2540…etc. in fig. 25 and corresponding analogues in figs. 26-27 can reasonably function as trigger, in response of which virtual image generation and/or 3D model of a captured subject is created in addition to outputting the virtual image, 3D model or combination thereof.
E.g., regarding block 2512, Curry discloses, ‘Next, the system will check at box 2512 to see if the quarterback rolled out into the pocket or not. This can be determined simply by the position of the ball in conjunction with the position of the quarterback and whether or not the two are moving together behind the line of scrimmage.’ Here, the check box 2512 meets the limitation, movement state of the subject and/or posture of the subject
Regarding check box 2508, Curry discloses,
This can be determined at the beginning of the play based on location data of the quarterback and the ball. If position information from the quarterback and the ball indicates that the quarterback is a few yards or more removed from the ball, then the system assumes the play is a shotgun formation.
Here, the QB being within few yards of the ball is understood as meeting limitation of posture of the subject, or movement state of the subject.
Then Curry states, If so, the system will operate such that the vantage point of the viewer will zoom and track from the EWS into a Wide Shot 2510 of the Quarterback once the ball is snapped – indicating live-action image capturing and output switching operation being undertaken following the trigger.
Likewise, other check boxes 2504, 2516, 2520, 2524, 2528, 2532, 2534, 2540…etc. in fig. 25 also understood pertains to trigger that falls in one or more of a number of captured images of the subject, a posture of the subject, or a movement state of the subject).
Regarding claim 2, Curry discloses the information processing device according to claim 1, wherein the virtual image includes an image of a viewpoint different from a viewpoint of the imaging device that captures the live-action image (¶0127-0128, fig. 20b, ¶0195, ¶0211-0218).
Regarding claim 3, Curry discloses the information processing device according to claim 1, wherein the circuitry selectively performs processing of outputting the live-action image and processing of outputting the virtual image (This would provide the viewer with a perspective of a live game that would look just like a CG sports video game of the event. The 3D model of the event created in software along with the 3D model of the ball and the CG avatar models of the players then interact together like a CG video game, ¶0128. Also see ¶0063, 0127-0128. See fig 20b, ¶0195, ¶211-0218).
Regarding claim 4, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of automatically determining the trigger on a basis of input information and outputting the virtual image (These viewing options are where the software takes over the director's chair and picks from a variety of shots based on what is occurring in the game itself., ¶0218. Also see ¶0271-0272.
See fig 20b, ¶0185-087, ¶0195, ¶211-0218).
Regarding claim 9, Curry discloses the information processing device according to claim 1, wherein circuitry performs processing of determining a predetermined operation input as occurrence of the trigger, setting a viewpoint position on a basis of a viewpoint position of the live-action image (This then identifies to the system the 2D viewer vantage point desired or the 3D location where the live video feed (video signal) should be shot from in the 3D event arena. At block 1218, the software composes the desired live video shot or 2D viewer vantage point using the array of feedback cameras in the camera network., ¶0126.
The camera feed then transitions to the zoomed-in image captured by camera 1620 at the vantage point 1640. The camera feed then transitions to the zoomed-in image captured by camera 1618 at the vantage point 1642. The camera feed then transitions to the zoomed-in image captured by camera 1616 at the vantage point 1644. The camera feed then transitions to the zoomed-in image captured by camera 1614 at the vantage point 1646., ¶0152.
The viewer vantage point is able to track anywhere in the 3D environment and their 2D perspective is rendered in real-time, ¶0212
the present system operates in real time for camera selection and viewpoint selection, ¶0247), and outputting the virtual image (¶0124-0129. See fig 20b, ¶0195, ¶211-0218.
If no fumble occurred, the next conditional check at box 2508 of FIG. 25 is whether or not a shotgun snap was used. This can be determined at the beginning of the play based on location data of the quarterback and the ball. If position information from the quarterback and the ball indicates that the quarterback is a few yards or more removed from the ball, then the system assumes the play is a shotgun formation. If so, the system will operate such that the vantage point of the viewer will zoom and track from the EWS into a Wide Shot 2510 of the Quarterback once the ball is snapped. The angle will be on a diagonal facing the offense, ten yards away from the ball and ten yards up the field from the offense's perspective. This shot can also be chosen to be ten yards removed from the line of scrimmage behind the offense and looking in the direction the offense is moving, ¶0224).
Regarding claim 10, Curry discloses the information processing device according to claim 1, wherein circuitry performs processing of outputting the virtual image with a changed viewpoint in response to an operation input in a case where the virtual image is output (In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event, ¶0028.
See fig 20b, ¶0195, ¶211-0218).
Regarding claim 11, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of outputting the virtual image from a viewpoint of a specific subject in response to a designation operation of the specific subject in a case where the live-action image or the virtual image is output (The system can then choose vantage points from specific 3D locations from within this environment based on position information of the ball and the players in the game. The 3D location is then used to compose the shot from the camera arrays and interpolation techniques, ¶0274.
See fig 20b, ¶0195, ¶211-0218).
Regarding claim 12, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of outputting the virtual image while a predetermined operation input continues (The fourth system component is software that selects the output viewer vantage point given the position information inputs and the camera network feedback inputs. The functionality of the software is illustrated in the system execution steps (steps three to seven) shown in blocks 1208, 1210, 1212, 1214, 1216, 1218, and 1220. In the first step (1202), a 3D model of the event arena is constructed in software., ¶0125.
See fig 20b, ¶0195, ¶211-0218).
Regarding claim 13, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of displaying only a subject person related to a scene in the virtual image (In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event., ¶0128.
Also, the vantage point of the viewer may zoom and track around the player as he runs toward the end zone in slow motion, ¶0247.
See fig 20b, ¶0195, ¶211-0218).
Regarding claim 14, Curry discloses the information processing device according to claim 1, wherein the circuitry selectively performs processing of outputting the live-action image, processing of outputting the virtual image, and processing of outputting a live-action free viewpoint image using the three-dimensional model based on the live-action image (For viewing control, a end user then has four main viewing control options to select from: (1) Traditional Viewing Option (2) Manual Viewing Option (3) Player Manual Viewing Option (4) Mixed Viewing Option, ¶0191-0195.
The ball and player movements, however, are the same as what is occurring in the live event. CG audiences are also imposed in the stadium stands, for a more life-like experience. The viewer is then able to see this video-game like view of the event in either a pre-selected or manual viewing option, ¶0128.
In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event., ¶0128.
See fig 20b, ¶0195, ¶211-0218).
Regarding claim 15, Curry discloses the information processing device according to claim 1, wherein the image processing unit performs processing of outputting both the live-action image and the virtual image (The second way to display the event in the seventh step (1218), is to use 3D imaging technology to mix live images taken by the cameras with the 3D model of the game created in software, ¶0129).
Regarding claim 16, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of generating the virtual image based on the estimation information regarding the subject generated based of the captured image and the three-dimensional model of the subject (At block 1212, the player location data is then processed by the software, and the software creates 3D models of the players and the ball. The ball is constructed using at least three position data points and the player position information is fitted with 3D model avatars as described before., ¶0126).
Regarding claim 17, Curry discloses the information processing device according to claim 1, wherein the circuitry processing of generating the estimation information regarding the subject on a basis of the captured image (The result of the rows of camera arrays and interpolated images is the ability to track the viewer vantage point, ¶0155
The processor can also interpolate images in between cameras for more seamless transitions between cameras. The system performs such operations in accordance with the position information and other processing such as position estimation of balls, players, and equipment., ¶0269).
Regarding method claim(s) 18, although wording is different, the material is considered substantively equivalent to the device claim(s) 1 as described above.
Regarding claim(s) 19, although wording is different, the material is considered substantively equivalent to the device claim(s) 1 as described above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over Curry in view of Lin et al. (US 20150229883 A1, hereinafter Lin).
Regarding claim 5, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger and outputting the virtual image in a case where a predetermined scene is not shown in the live-action image (The second way to display the event in the seventh step (1218), is to use 3D imaging technology to mix live images taken by the cameras with the 3D model of the game created in software, and render a 2D viewer vantage point of those images. The multiple video signals are combined together in software using 3D imaging technology to provide the system with a 3D environment of vantage points to select from. There are two ways to implement the mixture of 3D models and live video signal information. Both implementations use the 3D model of the gaming arena and the CG 3D model avatars of the players. The first implementation attaches video signal information to the pre-existing 3D models, and updates it as it changes. The second implementation in addition creates 3D models of the players and game in real time. The second implementation then compares the real time 3D models with the pre-existing 3D models and augments its pre-existing 3D models. Then this method attaches the video signal information to the new 3D model. This second method can be used in conjunction with the Manual Viewing Option mentioned later. The second method can additionally be used to show pre-selected viewing options similar to those described herein, ¶0047).
Curry also discloses that predetermined scene could potentially be missed during live streaming (Some important moments of play can be missed if the selected broadcast feed is not focused on the action of play. Often, the action in a game or sporting event occurs so quickly that it is difficult or in some cases impossible given the limits of human reaction speed for human observation and decision making to keep up with optimal selection of cameras and vantage point, ¶0006).
However, Curry is not found disclosing expressly that outputting the virtual image in a case where a predetermined scene is not shown in the live-action image.
However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶0020).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is shown instead of the missing element as disclosed in Lin, to obtain, outputting the virtual image in a case where a predetermined scene is not shown in the live-action image, because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed (see Lin ¶0020).
Regarding claim 6, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger in accordance with an important scene determination result (Some important moments of play can be missed if the selected broadcast feed is not focused on the action of play. Often, the action in a game or sporting event occurs so quickly that it is difficult or in some cases impossible given the limits of human reaction speed for human observation and decision making to keep up with optimal selection of cameras and vantage point, ¶0006.
The ball's location at a given time or the real-time position information of the ball is determined. The ball in a game is constructed so that a position sensor such as a transceiver and/or accelerometer can fit into the ball itself (see FIGS. 1 through 3), ¶0050) and outputting the virtual image as a playback image (¶0128, 0129).
Curry is not found disclosing expressly that determining occurrence of the trigger in accordance with an important scene determination result and outputting the virtual image as a playback image.
However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶0020).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger in accordance with an important scene determination result and outputting the virtual image as a playback image, because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed (see Lin ¶0020).
Regarding claim 7, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger (It is also possible to expand this feature to allow the viewer to view the CG environment from pre-selected view points based on the position information of the ball and players. However, real video signals from these angles are likely more exciting and appealing to view than CG imagery. Although, the option to add pre-selected view points based on position information as the rest of the viewing options do, is another option that the invention can accomplish, ¶0213).
Curry is not found disclosing expressly that, determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger.
However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶0020).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger, because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed (see Lin ¶0020).
Regarding claim 8, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger and outputting the virtual image (¶0128-0129) in a case where a main subject to be displayed in the live-action image is not shown in the live-action image.
Curry is not found disclosing expressly that, determining occurrence of the trigger and outputting the virtual image in a case where a main subject to be displayed in the live-action image is not shown in the live-action image.
However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶0020).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger and outputting the virtual image in a case where a main subject to be displayed in the live-action image is not shown in the live-action image, because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed (see Lin ¶0020).
Allowable Subject Matter
Claim 20 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 20, prior arts of record taken alone or in combination fails to reasonably disclose or suggest
The information processing device according to claim 1, wherein the trigger includes a number of captured images of the subject without occlusion of the subject.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NURUN FLORA whose telephone number is (571)272-5742. The examiner can normally be reached M-F 9:30 am -5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571) 272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NURUN FLORA/Primary Examiner, Art Unit 2619
Prosecution Insights