CTFR 18/715,126 CTFR 90164 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Response to Amendment Claims 1-19 are pending. Claims 1, and 3-19 are amended. Response to Arguments 07-37 AIA Applicant's arguments filed 2/20/2026 have been fully considered but they are not persuasive. Applicant primarily argues that newly amended claim is not taught in Curry reference. Applicant argues , Without acquiescing in any way to the Examiner's characterization of the cited reference with respect to the previously recited subject matter, Applicant hereby amends the claims in an effort to expedite prosecution by clarifying patentable distinctions. Applicant respectfully submits that Curry clearly does not teach or suggest, inter alia, "circuitry configured to perform processing of outputting , in respon se to a trigger, at least o ne of a virtual image generated based on estimation information regarding a subject, the estim ation information being generated based on at least one of a captured image or sensor information, or a three-di mensional model of the subject in a case where a live-action image captured by an imaging device is output, wherein the output ting is switched between modes of outputt ing the virtual image, outputting the three-dimensional model, or outputting both the virtual image and the three-dimensional model in respons e to the triqqer," as recite d by amended independent claim 1 (emphasis added), and the Office Action does not assert otherwise. Examiner does not agree with Applicant’s arguments and conclusions drawn therefrom. E.g., in fig. 20b Curry clearly mentions selectable modes (selection is understood as trigger) 1-4, wherein mixed viewing option 2096 is a combination of options 1-3. Also, Options 1-3 could reasonably be understood as meeting limitation of outputting of a virtual image and/or a 3D model of a subject. E.g., option 2092 (or option 4) makes use of the first and second methods used to implement the seventh step 1218 of FIG. 12. In this viewing option, 3D environments of the game are constructed using the 3D model of the event created in software, and the player CG avatars created using the player position information . Option 2094 (or option 3) on the other hand allows the end user to transition through each of the POV shots of each player in the game one by one. Once the end user selects option 2094, they will next be referred to the end user game viewing options screen 2100 shown in FIG. 21. Therefore, option 2094 could reasonably meet the limitation regarding outputting of “virtual image generated based on estimation information regarding a subject”, while option 2092 can reasonably be understood as meeting the limitation of “a three-dimensional model of the subject”. Also, as described before, mixed viewing option 2096 can reasonably meet the limitation of “outputting both the virtual image and the three-dimensional model in response to the trigger.” For details see the rejection below . Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-4, and 9-19 is/are rejected under 35 U.S.C. 102( a)(1) and/or 102(a)(2 ) as being anticipated by Curry (US 20100026809 A1) . Regarding claim 1, Curry discloses an information processing device (fig. 8, ¶0006, 30a-d) comprising: circuitry ( Image processing techniques such as triangulation can be used to help detect the location of different objects such as the ball in a game of play , ¶0055) configured to perform processing of outputting, in response to a trigger, at least one of a virtual image generated based on of estimation information regarding a subject, the estimation information being generated based on at least one of a captured image or sensor information, or a three-dimensional model of the subject in a case where a live-action image captured by an imaging device is output ( The camera recorded the position of the actor's movements via the motion capture dots. Then the 3D CG character was fitted to positions recorded by the motion capture dots in the scene later. In a similar way then one embodiment (viewing mode) uses the position sensors to capture movements of the players. Then in software 3D CG avatars are fitted to the skeletal stick figures created by the multiple sensors. One method that can additionally be used to create the 3D CG character avatars is triangulation 3D laser scanning. This allows the system to get more precise 3D models of the players. The players motion points (skeletal and muscles) are still difficult to model completely inside software, ¶0063. Also see ¶0048, ¶0083, ¶0124-0133 FIG. 20B displays a sample end user Menu interface 2088 that the viewer can use to select the different types of viewing options. A first option is Traditional Viewing Option 2090, a second option is Manual Viewing Option 2092, a third option is Player Manual Viewing Option 2094, and a fourth option is Mixed Viewing Option 2096. – ¶0195 The Manual Viewing Option 2092 makes use of the first and second methods used to implement the seventh step 1218 of FIG. 12. In this viewing option, 3D environments of the game are constructed using the 3D model of the event created in software, and the player CG avatars created using the player position information. The first method then uses the 3D event in software for broadcasting the event. The second method combines live video signal imagery with the 3D game occurring from inside software. This is accomplished as described before using two different techniques. – ¶0211 The Player Manual Viewing Option 2094 described here is for the sport of American football. Likewise, the same methodology can be applied to other sports such as soccer, and basketball. This allows the end user to transition through each of the POV shots of each player in the game one by one. Once the end user selects option 2094, they will next be referred to the end user game viewing options screen 2100 shown in FIG. 21. – ¶0215), wherein the outputting is switched between modes of outputting the virtual image, outputting the three-dimensional model, or outputting both the virtual image and the three-dimensional model in response to the trigger (Options 1-3 in fig. 20b could reasonably be understood as meeting limitation of outputting of a virtual image and/or a 3D model of a subject. E.g., option 2092 (or option 4) makes use of the first and second methods used to implement the seventh step 1218 of FIG. 12. In this viewing option, 3D environments of the game are constructed using the 3D model of the event created in software, and the player CG avatars created using the player position information . Option 2094 (or option 3) on the other hand allows the end user to transition through each of the POV shots of each player in the game one by one. Once the end user selects option 2094, they will next be referred to the end user game viewing options screen 2100 shown in FIG. 21. Therefore, option 2094 could reasonably meet the limitation regarding outputting of “virtual image generated based on estimation information regarding a subject”, while option 2092 can reasonably be understood as meeting the limitation of “a three-dimensional model of the subject”. Also, as described before, mixed viewing option 2096 can reasonably meet the limitation of “outputting both the virtual image and the three-dimensional model in response to the trigger.” See fig 20b, ¶0195, ¶211-0218). Regarding claim 2, Curry discloses the information processing device according to claim 1, wherein the virtual image includes an image of a viewpoint different from a viewpoint of the imaging device that captures the live-action image (¶0127-0128, fig. 20b, ¶0195, ¶0211-0218 ). Regarding claim 3, Curry discloses the information processing device according to claim 1, wherein the circuitry selectively performs processing of outputting the live-action image and processing of outputting the virtual image ( This would provide the viewer with a perspective of a live game that would look just like a CG sports video game of the event. The 3D model of the event created in software along with the 3D model of the ball and the CG avatar models of the players then interact together like a CG video game, ¶0128. Also see ¶0063, 0127-0128. See fig 20b, ¶0195, ¶211-0218 ). Regarding claim 4, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of automatically determining the trigger on a basis of input information and outputting the virtual image ( These viewing options are where the software takes over the director's chair and picks from a variety of shots based on what is occurring in the game itself., ¶0218. Also see ¶0271-0272. See fig 20b, ¶0195, ¶211-0218). Regarding claim 9, Curry discloses the information processing device according to claim 1, wherein circuitry performs processing of determining a predetermined operation input as occurrence of the trigger, setting a viewpoint position on a basis of a viewpoint position of the live-action image ( This then identifies to the system the 2D viewer vantage point desired or the 3D location where the live video feed (video signal) should be shot from in the 3D event arena. At block 1218, the software composes the desired live video shot or 2D viewer vantage point using the array of feedback cameras in the camera network., ¶0126. The camera feed then transitions to the zoomed-in image captured by camera 1620 at the vantage point 1640. The camera feed then transitions to the zoomed-in image captured by camera 1618 at the vantage point 1642. The camera feed then transitions to the zoomed-in image captured by camera 1616 at the vantage point 1644. The camera feed then transitions to the zoomed-in image captured by camera 1614 at the vantage point 1646., ¶0152. The viewer vantage point is able to track anywhere in the 3D environment and their 2D perspective is rendered in real-time, ¶0212 the present system operates in real time for camera selection and viewpoint selection, ¶0247 ), and outputting the virtual image (¶ 0124-0129. See fig 20b, ¶0195, ¶211-0218). Regarding claim 10, Curry discloses the information processing device according to claim 1, wherein circuitry performs processing of outputting the virtual image with a changed viewpoint in response to an operation input in a case where the virtual image is output ( In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event , ¶ 0028. See fig 20b, ¶0195, ¶211-0218). Regarding claim 11, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of outputting the virtual image from a viewpoint of a specific subject in response to a designation operation of the specific subject in a case where the live-action image or the virtual image is output ( The system can then choose vantage points from specific 3D locations from within this environment based on position information of the ball and the players in the game. The 3D location is then used to compose the shot from the camera arrays and interpolation techniques , ¶ 0274. See fig 20b, ¶0195, ¶211-0218). Regarding claim 12, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of outputting the virtual image while a predetermined operation input continues ( The fourth system component is software that selects the output viewer vantage point given the position information inputs and the camera network feedback inputs . The functionality of the software is illustrated in the system execution steps (steps three to seven) shown in blocks 1208, 1210, 1212, 1214, 1216, 1218, and 1220. In the first step (1202), a 3D model of the event arena is constructed in software . , ¶ 0125. See fig 20b, ¶0195, ¶211-0218). Regarding claim 13, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of displaying only a subject person related to a scene in the virtual image ( In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event. , ¶ 0128 . Also, the vantage point of the viewer may zoom and track around the player as he runs toward the end zone in slow motion , ¶ 0247. See fig 20b, ¶0195, ¶211-0218). Regarding claim 14, Curry discloses the information processing device according to claim 1, wherein the circuitry selectively performs processing of outputting the live-action image, processing of outputting the virtual image, and processing of outputting a live-action free viewpoint image using the three-dimensional model based on the live-action image ( For viewing control, a end user then has four main viewing control options to select from: (1) Traditional Viewing Option (2) Manual Viewing Option (3) Player Manual Viewing Option (4) Mixed Viewing Option, ¶0191-0195. The ball and player movements, however, are the same as what is occurring in the live event . CG audiences are also imposed in the stadium stands, for a more life-like experience. The viewer is then able to see this video-game like view of the event in either a pre-selected or manual viewing option , ¶ 0128. In the manual viewing option the viewer is be able to manually track their viewer vantage point into any angle or 3D viewpoint of the CG event. , ¶ 0128. See fig 20b, ¶0195, ¶211-0218). Regarding claim 15, Curry discloses the information processing device according to claim 1, wherein the image processing unit performs processing of outputting both the live-action image and the virtual image ( The second way to display the event in the seventh step (1218), is to use 3D imaging technology to mix live images taken by the cameras with the 3D model of the game created in software , ¶ 0129 ). Regarding claim 16, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of generating the virtual image based on the estimation information regarding the subject generated based of the captured image and the three-dimensional model of the subject ( At block 1212, the player location data is then processed by the software, and the software creates 3D models of the players and the ball. The ball is constructed using at least three position data points and the player position information is fitted with 3D model avatars as described before. , ¶ 0126 ). Regarding claim 17, Curry discloses the information processing device according to claim 1, wherein the circuitry processing of generating the estimation information regarding the subject on a basis of the captured image ( The result of the rows of camera arrays and interpolated images is the ability to track the viewer vantage point , ¶ 0155 The processor can also interpolate images in between cameras for more seamless transitions between cameras. The system performs such operations in accordance with the position information and other processing such as position estimation of balls, players, and equipment. , ¶ 0269 ). Regarding method claim(s) 18, although wording is different, the material is considered substantively equivalent to the device claim(s) 1 as described above. Regarding claim(s) 19, although wording is different, the material is considered substantively equivalent to the device claim(s) 1 as described above . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over Curry in view of Lin et al. (US 20150229883 A1, hereinafter Lin) . Regarding claim 5, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger and outputting the virtual image in a case where a predetermined scene is not shown in the live-action image ( The second way to display the event in the seventh step (1218), is to use 3D imaging technology to mix live images taken by the cameras with the 3D model of the game created in software , and render a 2D viewer vantage point of those images. The multiple video signals are combined together in software using 3D imaging technology to provide the system with a 3D environment of vantage points to select from. There are two ways to implement the mixture of 3D models and live video signal information. Both implementations use the 3D model of the gaming arena and the CG 3D model avatars of the players. The first implementation attaches video signal information to the pre-existing 3D models, and updates it as it changes. The second implementation in addition creates 3D models of the players and game in real time. The second implementation then compares the real time 3D models with the pre-existing 3D models and augments its pre-existing 3D models. Then this method attaches the video signal information to the new 3D model. This second method can be used in conjunction with the Manual Viewing Option mentioned later. The second method can additionally be used to show pre-selected viewing options similar to those described herein, ¶ 0047 ). Curry also discloses that predetermined scene could potentially be missed during live streaming ( Some important moments of play can be missed if the selected broadcast feed is not focused on the action of play. Often, the action in a game or sporting event occurs so quickly that it is difficult or in some cases impossible given the limits of human reaction speed for human observation and decision making to keep up with optimal selection of cameras and vantage point , ¶ 0006 ). However, Curry is not found disclosing expressly that outputting the virtual image in a case where a predetermined scene is not shown in the live-action image. However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶ 0020 ). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is shown instead of the missing element as disclosed in Lin, to obtain, outputting the virtual image in a case where a predetermined scene is not shown in the live-action image, because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed ( see Lin ¶0020 ). Regarding claim 6, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger in accordance with an important scene determination result ( Some important moments of play can be missed if the selected broadcast feed is not focused on the action of play . Often, the action in a game or sporting event occurs so quickly that it is difficult or in some cases impossible given the limits of human reaction speed for human observation and decision making to keep up with optimal selection of cameras and vantage point , ¶ 0006 . The ball's location at a given time or the real-time position information of the ball is determined. The ball in a game is constructed so that a position sensor such as a transceiver and/or accelerometer can fit into the ball itself (see FIGS. 1 through 3) , ¶ 0050 ) and outputting the virtual image as a playback image (¶0128, 0129 ). Curry is not found disclosing expressly that determining occurrence of the trigger in accordance with an important scene determination result and outputting the virtual image as a playback image . However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶ 0020 ). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger in accordance with an important scene determination result and outputting the virtual image as a playback image , because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed ( see Lin ¶0020 ). Regarding claim 7, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger ( It is also possible to expand this feature to allow the viewer to view the CG environment from pre-selected view points based on the position information of the ball and players. However, real video signals from these angles are likely more exciting and appealing to view than CG imagery. Although, the option to add pre-selected view points based on position information as the rest of the viewing options do, is another option that the invention can accomplish, ¶ 0213 ). Curry is not found disclosing expressly that, determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger. However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶ 0020 ). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger as a determination result of a selection condition for the live-action image or the virtual image, and outputting the virtual image in accordance with the determination for the occurrence of the trigger , because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed (see Lin ¶0020 ). Regarding claim 8, Curry discloses the information processing device according to claim 1, wherein the circuitry performs processing of determining occurrence of the trigger and outputting the virtual image (¶0128-0129) in a case where a main subject to be displayed in the live-action image is not shown in the live-action image. Curry is not found disclosing expressly that, determining occurrence of the trigger and outputting the virtual image in a case where a main subject to be displayed in the live-action image is not shown in the live-action image. However, Lin discloses that avatar of a user’s face is displayed when the face is missing in the live video stream (¶ 0020 ). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA) to modify the invention of Curry such that in case an important scene is not shown in the live-action image, a virtual image is played back instead of the missing element as disclosed in Lin, to obtain, determining occurrence of the trigger and outputting the virtual image in a case where a main subject to be displayed in the live-action image is not shown in the live-action image , because, combining prior art elements ready to be improved according to known method to yield predictable results is obvious. Furthermore, such combination would yield natural transition in the scene sequence wherein the important subject might be missing in the live feed ( see Lin ¶0020 ). Conclusion 07-40 AIA Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL . See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NURUN FLORA whose telephone number is (571)272-5742. The examiner can normally be reached M-F 9:30 am -5:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571) 272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /NURUN FLORA/Primary Examiner, Art Unit 2619 Application/Control Number: 18/715,126 Page 2 Art Unit: 2619 Application/Control Number: 18/715,126 Page 3 Art Unit: 2619 Application/Control Number: 18/715,126 Page 4 Art Unit: 2619 Application/Control Number: 18/715,126 Page 5 Art Unit: 2619 Application/Control Number: 18/715,126 Page 6 Art Unit: 2619 Application/Control Number: 18/715,126 Page 7 Art Unit: 2619 Application/Control Number: 18/715,126 Page 8 Art Unit: 2619 Application/Control Number: 18/715,126 Page 9 Art Unit: 2619 Application/Control Number: 18/715,126 Page 10 Art Unit: 2619 Application/Control Number: 18/715,126 Page 11 Art Unit: 2619 Application/Control Number: 18/715,126 Page 12 Art Unit: 2619 Application/Control Number: 18/715,126 Page 13 Art Unit: 2619 Application/Control Number: 18/715,126 Page 14 Art Unit: 2619 Application/Control Number: 18/715,126 Page 15 Art Unit: 2619 Application/Control Number: 18/715,126 Page 16 Art Unit: 2619
Prosecution Insights